Telemedicine System With Dynamic Imaging

ABSTRACT

A telemedicine system with dynamic imaging is disclosed herein. In some embodiments, the telemedicine system comprises a laser imaging and treatment apparatus, and associated systems and methods that allow a physician (e.g., a surgeon) to perform laser surgical procedures on an eye structure or a body surface with a laser imaging and treatment apparatus disposed at a first (i.e. local) location from a control system disposed at a second (i.e. remote) location, e.g., a physician&#39;s office. Also, in some embodiments, communication between the laser imaging and treatment apparatus and control system is achieved via the Internet®. Further, in some embodiments, the telemedicine system includes a dynamic imaging system and/or a facial recognition system that verifies the identity of a patient, and is capable of being used for other important applications, such as tracking and analyzing trends in a disease process.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a continuation-in-part of U.S. patentapplication Ser. No. 16/666,230, filed Oct. 28, 2019; and U.S. patentapplication Ser. No. 16/666,230 is a continuation-in-part of U.S. patentapplication Ser. No. 15/897,085, filed Feb. 14, 2018, now U.S. Pat. No.10,456,209; which claims the benefit of U.S. Provisional Application No.62/459,009, filed on Feb. 14, 2017, U.S. Provisional Application No.62/549,941, filed on Aug. 24, 2017, and to U.S. Provisional ApplicationNo. 62/563,582, filed on Sep. 26, 2017; and U.S. patent application Ser.No. 15/897,085 is a continuation-in-part of U.S. patent application Ser.No. 15/368,759, filed Dec. 5, 2016, now U.S. Pat. No. 9,931,171; whichis a continuation-in-part of U.S. patent application Ser. No.14/715,325, filed May 18, 2015, now U.S. Pat. No. 9,510,974; which is acontinuation-in-part of U.S. patent application Ser. No. 13/865,901,filed Apr. 18, 2013, now U.S. Pat. No. 9,037,217; which is acontinuation-in-part of U.S. patent application Ser. No. 13/573,100,filed Aug. 20, 2012, now U.S. Pat. No. 8,903,468; which is acontinuation-in-part of U.S. patent application Ser. No. 12/925,518,filed Oct. 22, 2010, now U.S. Pat. No. 8,452,372; which claims thebenefit of U.S. Provisional Application No. 61/455,111, filed Oct. 13,2010; the disclosure of each of which is hereby incorporated byreference as if set forth in their entirety herein.

U.S. patent application Ser. No. 16/666,230 is also acontinuation-in-part of Patent Cooperation Treaty Patent Application No.PCT/US2018/041958, filed Jul. 13, 2018; which claims the benefit of U.S.Provisional Application No. 62/532,098, filed on Jul. 13, 2017, U.S.Provisional Application No. 62/671,525, filed on May 15, 2018, U.S.Provisional Application No. 62/549,941, filed on Aug. 24, 2017, and toU.S. Provisional Application No. 62/563,582, filed on Sep. 26, 2017; thedisclosure of each of which is hereby incorporated by reference as ifset forth in their entirety herein.

FIELD OF THE INVENTION

The present invention relates to methods and systems for telemedicineand for laser treatment of the eye or a body surface. More particularly,the present invention relates to a telemedicine system with dynamicimaging and/or facial recognition capabilities.

BACKGROUND OF THE INVENTION

As is well known in the art, various eye disorders, such as diabeticretinopathy, vascular occlusion, neovascularization and age maculardegeneration, can, and in most instances will, have an adverse effect onthe retina. Indeed, if not treated at the appropriate stage, noteddiseases, particularly, diabetic retinopathy, can lead to severe lossesin vision.

Various methods and systems have thus been developed to aid in thediagnosis of the noted eye diseases. The method often employed by an eyecare specialist, such as an ophthalmologist, is to examine the ocularfundus (the inside back surface of the eye containing the retina, bloodvessels, nerve fibers, and other structures) with an ophthalmoscope.

The ophthalmoscope is a small, hand-held device, which, whenappropriately positioned, shines light through a subject's pupil toilluminate the fundus. By properly focusing the light reflected from thesubject's fundus, an examiner can observe the fundus structures.

As is well known in the art, examination of the ocular fundus can alsobe achieved using a fundus or slit lamp camera. Illustrative are theapparatus and systems disclosed in U.S. Pat. Nos. 5,713,047, 5,943,116,5,572,266, 4,838,680, 6,546,198, 6,636,696, 4,247,176, 5,742,374 and6,296,358.

Various method and systems have also been developed to treat eyedisorders, such as diabetic retinopathy, glaucoma and age maculardegeneration. One known method of treating the noted eye disorders, aswell as retinal detachment, is laser coagulation of predeterminedbiological structures of the eye, such as the retina.

As is well known in the art, during laser coagulation of an eyestructure, laser energy is transmitted to the structure to effectcoagulation thereof. A typical laser coagulation system thus includes alaser energy or beam source, such as a beam projector, a slit imageprojector or lamp for forming a slit image on the eye, and observationequipment for observing the slit image and laser spot(s) in the eye.Illustrative are the laser coagulation systems disclosed in U.S. Pat.Nos. 4,759,360 and 4,736,744.

A major drawback associated with each of the noted conventional systems,as well as most known laser coagulation systems (and associatedmethods), is that the conventional slit lamp systems require a contactlens to neutralize the refractive power of the cornea. A contact lens isalso necessary to provide a variable field of view of the retina up to130°.

As is well known in the art, the contact lens must be appropriatelypositioned on the surface of the cornea and held at the desired positionby the specialist, e.g., surgeon, while looking through the slit lampmicroscope.

During this conventional laser coagulation procedure, the contact lensis positioned on the cornea, and held in position by the surgeon so asto permit the surgeon to view the retina through the slit lampmicroscope during the laser application to the retina. In allconventional contact systems, the field of view is limited (e.g.,maximum 50-60 degrees) and the surgeon is required to move the contactlens from one side of the eye to the other side of the eye during theprocedure, and the patient is also required to move his or her eye, inorder to permit the surgeon to see the peripheral retina.

There are several drawbacks associated with the use of a contact lensduring laser coagulation. A major drawback is that the use of a contactlens requires topical anesthesia and a dilated pupil for laserapplication. As is well known in the art, a contact lens can, and inmany instances will, cause corneal abrasion on an anesthetized cornea.

A further drawback associated with conventional laser coagulationprocedures is that the laser procedures are dependent on the steadinessof the physician's hands and the subject's head.

Another apparatus that is often used for laser energy delivery to theperipheral retina is the indirect ophthalmoscope. Use of the indirectophthalmoscope requires a physician to hold an appropriate convex lensin front of the eye (pupil) with one hand to focus the laser beam on theretina, while the eye is indented with another hand to bring theperipheral retina into the field of view for laser application.

In the indirect ophthalmoscopy technique, which is an alternativeconventional method, the physician (i.e., surgeon) does not place acontact lens on the cornea, but rather he or she has to indent theperipheral part of the eye with an indenter (or scleral depressor) tobring the peripheral retinal areas into view, and additionally, thepatient has to move the eye side to side.

Although laser delivery with an indirect ophthalmoscope eliminates theneed for a contact lens, there are still drawbacks and disadvantagesassociated with use of an indirect ophthalmoscope. A major drawback isthat during laser delivery (and, hence, coagulation of a desired eyestructure), the ophthalmoscope is often carried on the physician's headfor 30-60 mins. This extended period causes extreme fatigue for thephysician.

The indentation of the eye for the extended period is also veryunpleasant for the subject or patient.

A further drawback associated with the use of an indirect ophthalmoscopefor laser coagulation is that the indirect ophthalmoscope does notprovide a retained record or documentation for future evaluation.Further, in most instances, the subject typically requires subsequentfundus photography.

None of the above described conventional methods are suitable for remotelaser application because they are limited in their field of view(typically 50-60 degrees). Also, the eye movement that is needed withthese systems to view the entire retina renders them unsuitable forremote applications.

It would thus be desirable to provide non-contact systems and methodsfor laser coagulation of eye structures to treat eye disorders, and arecapable of being effectively utilized to treat patients located at aremote site.

It is therefore an object of the present invention to providenon-contact systems and methods for laser coagulation of eye structuresthat substantially reduce or overcome the noted drawbacks anddisadvantages associated with conventional contact-based lasercoagulation systems and methods.

It is another object of the present invention to provide non-contactapparatus, systems and methods for laser imaging and coagulation of aneye structure.

It is yet another object of the present invention to provide non-contactapparatus, systems and methods for laser imaging and coagulation of theretina and its periphery to treat retina and choroideal disorders and/ordiseases.

It is still another object of the present invention to provide a systemfor remote laser treatment of an eye structure or a body surface that isfurther capable of performing photodynamic therapy on a patient.

It is yet another object of the present invention to provide atelemedicine system or a remote laser treatment system with dynamicimaging that more accurately verifies the identity of a patient, and iscapable of being used for other important applications, such as trackingand analyzing trends in a disease process.

It is still another object of the present invention to provide a remotelaser treatment system that determines the geographical location of thelocal laser generation unit of the system so as to ensure that the locallaser generation unit has not been improperly intercepted or stolen byunauthorized individuals.

SUMMARY OF THE INVENTION

The present invention is directed to laser imaging and coagulationapparatus, systems and methods that allow an eye specialist, e.g., anophthalmologist or surgeon, to perform laser surgery on an eyestructure, e.g. retina, with an integral laser imaging and coagulationapparatus disposed at a first (i.e. local) location from a controlsystem disposed at a second (i.e. remote) location, e.g., a physician'soffice. The laser imaging and coagulation apparatus, systems and methodsof the invention thus make it possible for an ophthalmologist to screenand perform laser surgery to treat various eye disorders, including,without limitation, diabetic retinopathy, vascular occlusion,neovascularization and age macular degeneration from a geographicallyremote location. The laser imaging and treatment system described hereinmay also be used to perform photodynamic therapy on a patient.

In one embodiment of the invention, the laser coagulation systemincludes

(i) at least a first laser-imaging system disposed at a first location,the first laser-imaging system including a first laser-imagingapparatus, a photoacoustic system, a first processor and a local controlmodule,

the first laser-imaging apparatus including a wide angle digital imageacquisition system for acquiring digital images of a subject's eyestructure and a laser generation system for transmitting an aiming laserbeam and at least a first coagulation laser beam to the eye structure,the first coagulation laser beam having a first laser energy,

the photoacoustic system being configured to measure temperature of eyestructure tissue subjected to the first laser energy,

the local control module including local operation, local operation andperformance simulation and local safety and verification sub-modules;and

(ii) a central control system disposed at a remote site, the centralcontrol system including a second processor and a remote control module,

the remote control module including remote operation, remote operationand performance simulation, and remote safety and verificationsub-modules,

the remote operations sub-module being configured to facilitatecommunications between a remote physician and the remote processor, andperform a laser coagulation procedure on the eye structure in an actualcontrol mode.

In some embodiments of the invention, the local operation sub-module isconfigured to acquire at least a first eye structure image from thedigital image acquisition system and transmit the first eye structureimage to the remote site, receive a target laser transmission area andlaser transmission parameters from a remote physician, apply an activecontour algorithm to partition the first eye structure image into a gridmap, perform a scatter laser (focal or grid) coagulation of the eyestructure under the remote physician's command, acquire a plurality ofpost-procedure eye structure images, and transmit the post-procedure eyestructure images to the remote site for evaluation and verification oftreatment.

In some embodiments, the remote operations sub-module is furtherconfigured to execute a virtual treatment of the eye structure andperform a test surgical procedure in association with the localoperation and performance simulation sub-module.

In some embodiments, the remote operation and performance simulationsub-module is configured to test performance parameters of the localoperation module and perform virtual treatment of the eye structure bythe remote physician.

In some embodiments of the invention, the photoacoustic system isconfigured to control the laser generation system.

In one embodiment of the invention, the eye structure comprises theretina.

In some embodiments of the invention, the laser coagulation system alsoincludes eye tracking means for tracking movement of the eye.

In some embodiments of the invention, the laser coagulation system alsoincludes facial recognition means for identifying and/or verifying theidentity of the subject.

In one embodiment, communication by and between the central controlsystem and the laser-imaging apparatus is achieved via the Internet®.

In another embodiment, the laser coagulation system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location;

at least a first laser-imaging system disposed at the first location,the laser-imaging system including a laser-imaging apparatus, a firstprocessor and a local control module;

the laser-imaging apparatus including a digital image acquisition systemconfigured to acquire digital images of the eye structure or the bodysurface, the local control module including local operation, localoperation and performance simulation, and local safety and verificationsub-modules;

a laser generation system configured to generate and transmit at least afirst aiming laser beam and at least a first coagulation laser beam, andmeans for controlling the digital image acquisition system and the lasergeneration system;

a central control system disposed at the remote site, the centralcontrol system including a second processor and a remote control module,the remote control module including remote operation, remote operationand performance simulation, and remote safety and verificationsub-modules; and

the remote operation sub-module being configured to facilitatecommunications between a remote physician and the second processor, andperform a laser coagulation procedure on the eye structure or the bodysurface in an actual control mode, the remote operation sub-moduleincluding a touchscreen interface configured to enable the remotephysician to draw a target laser treatment area or areas on a digitizedimage of the eye structure or the body surface.

In yet another embodiment, the laser coagulation system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

at least a first laser-imaging system disposed at the first location,the laser-imaging system including a laser-imaging apparatus, a lasergeneration system, a first computing device with a first processor, anda local control module;

the laser-imaging apparatus including a digital image acquisition systemconfigured to acquire a digitized image of the eye structure or the bodysurface, the local control module including local operation, localoperation and performance simulation, and local safety and verificationsub-modules, the local operation sub-module configured to acquire thedigitized image of the eye structure or the body surface from thedigital image acquisition system and transmit the digitized image to theremote site;

the laser generation system including an aiming laser configured togenerate and transmit an aiming laser beam to the eye structure or thebody surface, and a treatment laser configured to generate and transmitat least a first coagulation laser beam to the eye structure or the bodysurface;

the central control system including a second computing device with asecond processor, and a remote control module, the remote control moduleincluding remote operation, remote operation and performance simulation,and remote safety and verification sub-modules; and

the remote operation sub-module being configured to facilitatecommunications between a remote physician and the second processor ofthe second computing device, and perform a laser coagulation procedureon the eye structure or the body surface in an actual control mode inwhich the treatment laser is configured to transmit the firstcoagulation laser beam to the eye structure or the body surface.

In some embodiments of the invention, the first laser-imaging systemfurther includes an image recognition sensor configured to captureimages of a patient at the first location so that an identity of thepatient or an identity of a body portion of the patient is capable ofbeing identified and verified prior to the laser coagulation procedurebeing performed on the eye structure or the body surface in the actualcontrol mode.

In some embodiments, the image recognition sensor is operatively coupledto the first computing device, the first computing device beingspecially programmed to compare a first reference digital image of thepatient captured by the image recognition sensor at a first time to asecond digital image of the patient captured by the image recognitionsensor at a second subsequent time, and to determine if the seconddigital image of the patient matches or substantially matches the firstreference digital image of the patient (i.e., the second digital imageof the patient substantially matches the first reference digital imageof the patient when there are only minor differences between the twoimages, e.g., a blemish on the face of patient that appears in thesecond digital image, but not in the first reference digital image).

In some embodiments, when the first computing device determines that thesecond digital image of the patient substantially matches the firstreference digital image of the patient, the first computing device isspecially programmed to generate a matched image confirmationnotification that is sent to the second computing device at the remotesite in order to inform the remote physician that the patient has beenidentified and verified; and, when the first computing device determinesthat the second digital image of the patient does not substantiallymatch the first reference digital image of the patient, the firstcomputing device is specially programmed to generate a non-matchingimage notification that is sent to the second computing device at theremote site in order to inform the remote physician that the patient hasnot been properly identified and verified.

In some embodiments, when the first computing device determines that thesecond digital image of the patient does not substantially match thefirst reference digital image of the patient, the first computing deviceis further specially programmed to automatically lock out the treatmentlaser so that the treatment laser is not capable of being fired.

In some embodiments, the image recognition sensor is in the form of amultispectral camera configured to capture the images of the patientusing both visible light and infrared light.

In some embodiments, the first laser-imaging system further includes avoice recognition sensor configured to capture speech waveformsgenerated by the patient at the first location so that an identity ofthe patient is capable of being identified and verified prior to thelaser coagulation procedure being performed on the eye structure or thebody surface in the actual control mode.

In some embodiments, the voice recognition sensor is operatively coupledto the first computing device, the first computing device beingspecially programmed to compare a first reference speech waveform of thepatient captured by the voice recognition sensor at a first time to asecond speech waveform of the patient captured by the voice recognitionsensor at a second subsequent time, and to determine if the secondspeech waveform of the patient matches or substantially matches thefirst reference speech waveform of the patient (i.e., the second speechwaveform of the patient substantially matches the first reference speechwaveform of the patient when there are only minor differences betweenthe two speech waveforms, e.g., a minor difference in the tone of thespeech).

In some embodiments, when the first computing device determines that thesecond speech waveform of the patient matches or substantially matchesthe first reference speech waveform of the patient, the first computingdevice is specially programmed to generate a matched speech confirmationnotification that is sent to the second computing device at the remotesite in order to inform the remote physician that the patient has beenidentified and verified; and, when the first computing device determinesthat the second speech waveform of the patient does not match orsubstantially match the first reference speech waveform of the patient,the first computing device is specially programmed to generate anon-matching speech notification that is sent to the second computingdevice at the remote site in order to inform the remote physician thatthe patient has not been properly identified and verified.

In some embodiments, when the first computing device determines that thesecond speech waveform of the patient does not substantially match thefirst reference speech waveform of the patient, the first computingdevice is further specially programmed to automatically lock out thetreatment laser so that the treatment laser is not capable of beingfired.

In some embodiments, the voice recognition sensor is in the form of amicrophone configured to capture the speech waveforms generated by thepatient over a speech frequency range between 50 Hertz and 5,000 Hertz.

In still another embodiment, the laser treatment system includes: alocal control system disposed at a first location and a central controlsystem disposed at a remote site, the remote site being at a secondlocation, the local control system being operatively coupled to thecentral control system by means of a computer network; at least a firstlaser-imaging system disposed at the first location, the laser-imagingsystem including a laser-imaging apparatus, a laser generation system, afirst computing device with a first processor, and a local controlmodule;

the laser-imaging apparatus including a digital image acquisition systemconfigured to acquire a digitized image of the eye structure or the bodysurface, the local control module including local operation, localoperation and performance simulation, and local safety and verificationsub-modules, the local operation sub-module configured to acquire thedigitized image of the eye structure or the body surface from thedigital image acquisition system and transmit the digitized image to theremote site;

the laser generation system including an aiming laser configured togenerate and transmit an aiming laser beam to the eye structure or thebody surface, and a treatment laser configured to generate and transmitat least a first treatment laser beam to the eye structure or the bodysurface;

the central control system including a second computing device with asecond processor, and a remote control module, the remote control moduleincluding remote operation, remote operation and performance simulation,and remote safety and verification sub-modules; and

the remote operation sub-module being configured to facilitatecommunications between a remote physician and the second processor ofthe second computing device, and perform a laser treatment procedure onthe eye structure or the body surface in an actual control mode in whichthe treatment laser is configured to transmit the first treatment laserbeam to the eye structure or the body surface so as to surgically alterthe eye structure or the body surface.

In some embodiments of the invention, the first laser-imaging systemfurther includes an image recognition sensor configured to captureimages of a patient at the first location so that an identity of thepatient or an identity of a body portion of the patient is capable ofbeing identified and verified prior to the laser treatment procedurebeing performed on the eye structure or the body surface in the actualcontrol mode.

In some embodiments, the image recognition sensor is operatively coupledto the first computing device, the first computing device beingspecially programmed to compare a first reference digital image of thepatient captured by the image recognition sensor at a first time to asecond digital image of the patient captured by the image recognitionsensor at a second subsequent time, and to determine if the seconddigital image of the patient substantially matches the first referencedigital image of the patient.

In some embodiments, the image recognition sensor is in the form of aholoscopic three-dimensional camera configured to capture the images ofthe patient in three-dimensional form, and wherein the second computingdevice comprises a graphical user interface in the form of a multiview,three-dimensional visual display device configured to enable the remotephysician or another observer at the remote site to perform athree-dimensional analysis of the three-dimensional images of thepatient that are produced as a result of the patient being instructed toperform a task that alters one or more detectable physical attributes ofthe patient.

In some embodiments, the first computing device is specially programmedto utilize the three-dimensional images so as to collectively take intoaccount a plurality of physiological characteristics of a body area ofthe patient that reflect the one or more altered physical attributeswhen the identity of the patient or the identity of the body portion ofthe patient is identified and verified.

In some embodiments, the multiview, three-dimensional visual displaydevice of the second computing device is in the form of athree-dimensional digital holographic display device.

In some embodiments, the three-dimensional digital holographic displaydevice comprises one or more thin or ultrathin holographic opticalelements for producing high-resolution three-dimensional images, andwherein the three-dimensional digital holographic display devicecomprises an autostereoscopic three-dimensional display to eliminate theneed for the physician or the another observer to wear special eyewearwhile performing the three-dimensional analysis of the three-dimensionalimages of the patient.

In some embodiments, the multiview, three-dimensional visual displaydevice of the second computing device is in the form of a volumetricthree-dimensional display so as to generate the three-dimensional imagesof the patient formed by voxels with spatial depth and volume.

In some embodiments, the second computing device comprises a graphicaluser interface in the form of virtual reality glasses worn by the remotephysician or another observer at the remote site, the virtual realityglasses configured to enable the remote physician or the anotherobserver at the remote site to perform an analysis of the images of thepatient that are produced as a result of the patient being instructed toperform a task that alters one or more detectable physical attributes ofthe patient.

In some embodiments, the laser treatment system further comprises anoptical coherence tomography imaging system, near-infrared opticaltomography imaging system, or a frequency modulated continuous waveimaging system operatively coupled to the first computing device, theoptical coherence tomography imaging system, near-infrared opticaltomography imaging system, or frequency modulated continuous wave systemconfigured to capture additional images of the patient to supplement theimages of the patient captured by the image recognition sensor.

In some embodiments, the image recognition sensor is in the form oftwo-spaced apart cameras configured to capture the images of thepatient, and wherein the second computing device comprises a graphicaluser interface in the form of a head-mounted display device configuredto generate two display images, each of the two display images being infront of a respective one of the right and left eyes of the remotephysician or another observer at the remote site and corresponding tothe images of the patient captured by the two-spaced apart cameras.

In some embodiments, the image recognition sensor is in the form of athree-dimensional multi-color meta-holography device configured tocapture the images of the patient.

In some embodiments, the laser treatment system further comprises aphotoacoustic system being operatively coupled to the first computingdevice, the photoacoustic system including an ultrasound transducerconfigured to detect acoustic waves that are generated as a result ofthe absorption of energy by the eye structure or the body surface suchthat the photoacoustic system is able to capture ultrasonicthree-dimensional images of body structures beneath the skin of thepatient, the body structures beneath the skin of the patient includingbone structures of the patient.

In some embodiments, the treatment laser of the laser generation systemis configured to provide photodynamic therapy to the patient by emittinglight of a predetermined wavelength that is absorbed by tissue of a bodyportion of the patient to which a photosensitizer has been applied, thebody portion of the patient comprising a cancerous tumor, and thephotodynamic therapy configured to treat the cancerous tumor by killingthe cells forming the cancerous tumor.

In some embodiments, the photosensitizer is applied to the tissue of abody portion of the patient comprising the cancerous tumor by using aplurality of nanoparticles, and wherein the light emitted by thetreatment laser of the laser generation system is further absorbed bythe nanoparticles.

In some embodiments, the predetermined wavelength of the light emittedby the treatment laser of the laser generation system is betweenapproximately 380 nanometers and approximately 1550 nanometers.

In some embodiments, the laser treatment system further comprises adisplaceable prism or mirror disposed in the path of the first treatmentlaser beam emitted by the treatment laser, the displaceable prism ormirror being operatively coupled to the first computing device so thatthe displaceable prism or mirror is capable of being selectivelycontrolled by the first computing device based upon instructionsreceived from the second computing device at the remote site from theremote physician, the displaceable prism or mirror configured to enablethe first treatment laser beam to be applied to the tissue of thecancerous tumor of the patient in an oscillatory manner during thephotodynamic therapy.

In some embodiments, the light emitted by the treatment laser of thelaser generation system comprises ultraviolet light, and wherein thepower of the treatment laser is between approximately 2 milliwatts andapproximately 20 milliwatts.

In some embodiments, the digital image acquisition system of thelaser-imaging apparatus is configured to acquire a two-dimensional imageof the tissue of the cancerous tumor of the patient before, during, andafter the photodynamic therapy; and the laser treatment system furthercomprises a photoacoustic system being operatively coupled to the firstcomputing device, the photoacoustic system including an ultrasoundtransducer configured to detect acoustic waves that are generated as aresult of the absorption of energy by the tissue of the cancerous tumorof the patient such that the photoacoustic system is able to captureultrasonic three-dimensional images of the tissue of the cancerous tumorof the patient before, during, and after the photodynamic therapy.

In some embodiments, the photoacoustic system is further configured todetermine a temperature of the tissue of the cancerous tumor of thepatient subjected to laser energy from the first coagulation laser beam,the photoacoustic system further being configured to control the lasergeneration system by maintaining the laser energy of the first treatmentlaser beam at a predetermined energy level so as to prevent exceeding apredetermined threshold temperature during the photodynamic therapy.

In yet another embodiment, the laser treatment system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

the local control system including a laser generation device, a firstcomputing device with a first processor, a local control module, and adynamic imaging system;

the laser generation device including a treatment laser configured togenerate and transmit a treatment laser beam to a treatment location onor in the patient;

the central control system including a second computing device with asecond processor and a remote control module;

the remote control module being configured to perform a laser treatmentprocedure on the treatment location on or in the patient in an actualcontrol mode in which the treatment laser is configured to transmit thetreatment laser beam to the treatment location so as to surgically alterthe body of the patient in the treatment location; and

the dynamic imaging system including an imaging device operativelycoupled to the first computing device, the imaging device configured tocapture images of a body portion of the patient over a predeterminedduration of time so that a displacement of the body portion of thepatient is capable of being tracked during the predetermined duration oftime, the first computing device being programmed to determine thedisplacement of the body portion of the patient over the predeterminedduration of time using the captured images, and to compare thedisplacement of the body portion of the patient over the predeterminedduration of time to a reference displacement of the body portion of thepatient acquired prior to the displacement so that dynamic changes inthe body portion of the patient are capable of being assessed for thepurpose of identifying the patient or evaluating physiological changesin the body portion.

In still another embodiment, the telemedicine system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

the local control system including a first computing device with a firstprocessor, a local control module, and a dynamic imaging system;

the central control system including a second computing device with asecond processor and a remote control module;

the dynamic imaging system including an imaging device operativelycoupled to the first computing device, the imaging device configured tocapture images of a body portion of the patient over a predeterminedduration of time so that a displacement of the body portion of thepatient is capable of being tracked during the predetermined duration oftime, the first computing device being programmed to determine thedisplacement of the body portion of the patient over the predeterminedduration of time using the captured images, and to compare thedisplacement of the body portion of the patient over the predeterminedduration of time to a reference displacement of the body portion of thepatient acquired prior to the displacement so that dynamic changes inthe body portion of the patient are capable of being assessed for thepurpose of identifying the patient or evaluating physiological changesin the body portion.

In yet another embodiment, the telemedicine system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

the local control system including a first computing device with a firstprocessor, a local control module, and a facial recognition system;

the central control system including a second computing device with asecond processor and a remote control module;

the facial recognition system including an imaging device operativelycoupled to the first computing device, the imaging device configured tocapture one or more images of a face of the patient, the first computingdevice being programmed to compare one or more first reference digitalimages of the face of the patient captured by the imaging device offacial recognition system at a first time to one or more second digitalimages of the face of the patient captured by the imaging device of thefacial recognition system at a second subsequent time, and to determineif the one or more second digital images of the face of the patientsubstantially matches the one or more first reference digital images ofthe face of the patient.

In still another embodiment, the laser coagulation system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location, the local control system being operatively coupled tothe central control system by means of a computer network;

at least a first laser-imaging system disposed at the first location,the laser-imaging system including a laser-imaging apparatus, a lasergeneration system, a first computing device with a first processor, anda local control module;

the laser-imaging apparatus including a digital image acquisition andstorage system configured to take and store digital images of the eyestructure or the body surface, the digital image acquisition and storagesystem further including means for transmitting digital images;

the laser generation system including an aiming laser configured togenerate and transmit an aiming laser beam to the eye structure or thebody surface, and a treatment laser configured to generate and transmitat least a first coagulation laser beam to the eye structure or the bodysurface, and means for controlling the digital image acquisition andstorage system and the laser generation system;

the local control module including a local operation sub-module, a localoperation and performance simulation sub-module, the local operationsub-module configured to acquire at least a first digitized image of theeye structure or the body surface from the digital image acquisition andstorage system and transmit the first digitized image to the remotesite, the local operation sub-module further configured to receive atarget laser transmission area or areas and laser transmissionparameters from a remote physician, and determine a treatment area orpattern of spots on the first digitized image for application of thefirst coagulation laser beam, the local operation sub-moduleadditionally configured to perform a laser coagulation of the eyestructure or the body surface under the remote physician's command inwhich the first coagulation laser beam is applied to the treatment areaor each of the spots in the pattern, the local operation and performancesimulation sub-module configured to facilitate the testing of the systemprior to its operation in an actual control mode by replacing an actualeye structure or body surface of the subject with the first digitizedimage of the subject's eye structure or body surface;

the central control system being in communication with the laser-imagingapparatus, the central control system including a remote control moduleand a second computing device with a second processor configured toreceive and process command signals from the remote physician andtransmit the command signals to the local control module; and

the remote control module including a remote operation sub-moduleconfigured to facilitate communications between the remote physician andthe second processor of the second computing device, the remoteoperation sub-module, in association with the local operation andperformance simulation sub-module, further configured to execute avirtual treatment of the eye structure or the body surface, perform atest surgical procedure, and perform a fully automated and continuouslaser coagulation procedure over the entire area of the eye structure orthe body surface in the actual control mode in which the treatment laseris configured to transmit the first coagulation laser beam to the eyestructure or the body surface

In yet another embodiment, the laser coagulation system includes:

a local control system disposed at a first location and a centralcontrol system disposed at a remote site, the remote site being at asecond location;

at least a first laser-imaging system, the laser-imaging systemincluding a laser-imaging apparatus, a first processor and a localcontrol module;

the laser-imaging apparatus including a wide angle digital imageacquisition and storage system configured to take and store digitalimages of the retina, the digital image acquisition and storage systemincluding at least one retinal viewing camera that provides a field ofview of the retina in a range between 1600 and 200°, the digital imageacquisition and storage system further including means for transmittingdigital images;

a laser generation system configured to generate and transmit at least afirst aiming laser beam and at least a first coagulation laser beam, andmeans for controlling the digital image acquisition and storage systemand the laser generation system;

the local control module including a local operation sub-module, a localoperation and performance simulation sub-module, and a local safety andverification sub-module, the local operation sub-module configured toacquire at least a first retinal image of the retina from the digitalimage acquisition and storage system and transmit the first retinalimage to the remote site, the local operation sub-module furtherconfigured to receive a target laser transmission area and lasertransmission parameters from a remote physician, and determine a patternof spots on the first retinal image for application of the firstcoagulation laser beam, the local operation sub-module additionallyconfigured to perform a scatter laser coagulation of the retina underthe remote physician's command in which the first coagulation laser beamis applied to each of the spots in the pattern, acquire a plurality ofpost-procedure retinal images, and transmit the post-procedure retinalimages to the remote site for evaluation and verification of treatment,the local operation and performance simulation sub-module configured tofacilitate the testing of the system prior to its operation in an actualcontrol mode by replacing an actual eye of the subject with a digitizedfundus image of the subject's eye;

the central control system being in communication with the laser-imagingapparatus, and including a remote control module and a second processorconfigured to receive and process command signals from the remotephysician and transmit the command signals to the local control module;

the remote control module including a remote operation sub-moduleconfigured to facilitate communications between the remote physician andthe second processor, execute a virtual treatment of the retina, performa test surgical procedure, and perform a fully automated and continuouslaser coagulation procedure over the entire area of the retina in theactual control mode;

the remote control module further including a remote operation andperformance simulation sub-module and a remote safety and verificationsub-module, the remote operation and performance simulation sub-moduleconfigured to test performance parameters of the local operation moduleand perform a treatment simulation of the retina by the remote physicianwhile simulating eye movement of the subject by displacing the digitizedfundus image of the subject's eye in accordance with a plurality ofrandom variables;

the local and remote safety and verification sub-modules includingphysical, logical and medical safety constraints for safe operation ofthe system; and

wherein the system for laser coagulation of the retina is in the form ofa non-contact system that does not require the use of a contact lens.

BRIEF DESCRIPTION OF THE DRAWINGS

Further features and advantages will become apparent from the followingand more particular description of the preferred embodiments of theinvention, as illustrated in the accompanying drawings, and in whichlike referenced characters generally refer to the same parts or elementsthroughout the views, and in which:

FIG. 1 is an illustration of a human eye showing the major structuresthereof;

FIG. 2 is a schematic illustration of one embodiment of a laser-imagingapparatus, in accordance with the invention;

FIG. 3 is a schematic illustration of another embodiment of alaser-imaging apparatus, showing the elliptical mirror thereof, inaccordance with the invention;

FIG. 4 is a schematic illustration of a photoacoustic system, inaccordance with the invention;

FIG. 5 is a schematic illustration of one embodiment of a laser-imagingsystem, in accordance with the invention;

FIG. 6 is another schematic illustration of the laser-imaging systemshown in FIG. 5, showing the local and remote modules thereof, inaccordance with one embodiment of the invention;

FIG. 7 is an illustration of a human retina, showing an oval areaencompassing the target area on the retina for laser transmission, inaccordance with one embodiment of the invention;

FIG. 8 is an illustration of a human retina, showing the distribution oflaser spots resulting from a virtual treatment of the retina, inaccordance with one embodiment of the invention;

FIG. 9 is an illustration of a human retina, showing simulationvariables associated with movement of the eye, in accordance with oneembodiment of the invention;

FIG. 10 is an illustration of a human retina, showing a generated gridmap on the retina, in accordance with one embodiment of the invention;

FIG. 11 is an illustration of a human retina, showing a large area ofthe fundus that is to be coagulated, in accordance with one embodimentof the invention;

FIG. 12 is an illustration of a human retina, showing localized areas ofthe fundus that are to be coagulated, in accordance with one embodimentof the invention;

FIG. 13 is an illustration of a human retina, showing single, localizedspots of the fundus that are to be coagulated, in accordance with oneembodiment of the invention;

FIG. 14 is an illustration of a human face, showing an area of a skinlesion that is marked for laser application;

FIG. 15 is a schematic illustration of another embodiment of alaser-imaging system, in accordance with the invention, wherein thelocal control system of the laser-imaging system comprises image andvoice recognition sensors;

FIG. 16 is an illustration of a human face, showing exemplary areas ofthe human face that may be imaged so that the identity of a patient orthe identity of a body portion of a patient may be verified;

FIG. 17 is a schematic illustration of another embodiment of alaser-imaging system, in accordance with the invention, wherein theremote laser treatment system is provided with GPS-based identificationof the laser treatment application site;

FIG. 18a is an illustration of a human face, depicting a grid that isprojected over the facial area being analyzed using the dynamic imagingsystem of the remote laser treatment system described herein, andfurther depicting the positions of two exemplary points being used totrack dynamic changes in the lips of the person;

FIG. 18b is another illustration of a human face, depicting a grid thatis projected over the facial area being analyzed using the dynamicimaging system of the remote laser treatment system described herein,and further depicting the positions of two exemplary points being usedto track dynamic changes in the lips of the person while the person isspeaking the letter “O”;

FIG. 19a is an illustration of a finger being pressed against atransparent glass surface so that the tip of the finger is capable ofbeing imaged;

FIG. 19b depicts a finger before touching a transparent glass surfaceused for the imaging of the finger;

FIG. 19c depicts the finger touching the transparent glass surface, thefinger undergoing imaging that takes into account both surface andsubsurface properties of the finger in a two-dimensional and/orthree-dimensional manner;

FIG. 20 depicts an initial image of a face of a person captured by adynamic identity recognition system where the person has hair and facialhair;

FIG. 21 depicts another image of the face of the person in FIG. 20captured by the dynamic identity recognition system where the appearanceof the midsection of the face has remained generally stable, but dynamicchanges have occurred in the lower section of the face;

FIG. 22 depicts a computer-modified image of the face of the person inFIG. 20 where the hair and facial hair of the person have been removed,and the stable features of the person are depicted;

FIG. 23 depicts another image of the face of the person in FIG. 20captured by the dynamic identity recognition system where the appearanceof the midsection of the face has remained generally stable, but dynamicchanges have occurred in the lower section of the face;

FIG. 24 depicts a computer-modified image of the face of the person inFIG. 20 where the hair and facial hair of the person have been removed,and glasses and an earring have been added to the person;

FIG. 25 depicts a computer-modified image of the face of the person inFIG. 20 where an artificial change in the appearance of the person hasoccurred while the person is speaking one or more words;

FIG. 26 depicts a graphical representation of a correlation betweenmouth configuration and a sound wave generated by a person whilespeaking a series of words;

FIG. 27 depicts a graphical representation of a cognitive test fortesting a mental ability of a patient, wherein the testing objects areprovided right side up; and

FIG. 28 depicts another graphical representation of the cognitive testof FIG. 27, wherein the testing objects are provided upside down.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Before describing the present invention in detail, it is to beunderstood that this invention is not limited to particularlyexemplified apparatus, systems, structures or methods as such may, ofcourse, vary. Thus, although a number of apparatus, systems and methodssimilar or equivalent to those described herein can be used in thepractice of the present invention, the preferred apparatus, systems,structures and methods are described herein.

It is also to be understood that the terminology used herein is for thepurpose of describing particular embodiments of the invention only andis not intended to be limiting.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one having ordinaryskill in the art to which the invention pertains.

Further, all publications, patents and patent applications cited herein,whether supra or infra, are hereby incorporated by reference in theirentirety.

Finally, as used in this specification and the appended claims, thesingular forms “a, “an” and “the” include plural referents unless thecontent clearly dictates otherwise. Thus, for example, reference to “alaser image” includes two or more such images and the like.

Definitions

The terms “eye disorder” and “eye disease” are used interchangeablyherein and mean and include, without limitation, diabetic retinopathy,vascular occlusion, neovascularization, retinal detachment, neoplastictissue, ischemic retina, retinopathy of prematurity and age relatedmacular degeneration.

The following disclosure is provided to further explain in an enablingfashion the best modes of performing one or more embodiments of thepresent invention. The disclosure is further offered to enhance anunderstanding and appreciation for the inventive principles andadvantages thereof, rather than to limit in any manner the invention.The invention is defined solely by the appended claims including anyamendments made during the pendency of this application and allequivalents of those claims as issued.

As will readily be appreciated by one having ordinary skill in the art,the present invention substantially reduces or eliminates thedisadvantages and drawbacks associated with conventional systems andmethods for coagulating eye structures to treat eye disorders.

In overview, the present disclosure is directed to laser imaging andcoagulation apparatus, systems and methods that allow an eye specialist,e.g., an ophthalmologist or surgeon, to perform laser retinal surgicalprocedures, such as laser or tissue coagulation, with an integral laserimaging and coagulation apparatus disposed at a first (i.e. local)location from a control system disposed at a second (i.e. remote)location, e.g., a physician's office.

By the term “laser coagulation”, as used herein, it is meant to mean andinclude, without limitation, selective absorbance of transmitted lightenergy (having a visible green wavelength) by hemoglobin in biologicaltissue and, hence, sealing of blood vessels in the tissue. In apreferred embodiment of the invention, the wavelength of the transmittedenergy (referred to herein as a “treatment or coagulative laser beam”)is in the range of approximately 400-1650 nm, more preferably, in therange of approximately 530-670 nm.

Although the present invention is directed to thermotherapy ofbiological tissue by laser energy, it is to be understood the inventionis not limited to such form of energy. Indeed, as will readily beappreciated by one having ordinary skill in the art, the thermotherapyof biological tissue described herein, i.e. coagulation of selective eyestructures, can also be achieved via the application of electromagneticradiation, and radio frequency and ultrasound energy.

It is further to be understood that, although the biological tissuesubject to the thermotherapy (i.e. coagulation), in accordance with thepresent invention, comprises the retina, the invention is not limitedsolely to thermotherapy of the retina. According to the invention, thethermotherapy of the present invention can be employed to coagulate anyeye structure.

The laser-imaging apparatus, systems and methods of the invention, andlaser energy transmitted thereby, can thus be employed to treat variouseye disorders, including, without limitation, diabetic retinopathy,vascular occlusion, neovascularization, retinal detachment, neoplastictissue, ischemic retina, retinopathy of prematurity and age relatedmacular degeneration.

The laser-imaging apparatus, systems and methods of the invention, andlaser energy transmitted thereby, can also be readily employed inrefractive surgical procedures to, for example, perform corneal surfaceablation using an eximer or femtosecond laser, LASIK procedures, and/orlid surface and surrounding tissue tightening using an infrared laser.

The laser-imaging apparatus, systems and methods of the invention, andlaser energy transmitted thereby, can also be readily employed incosmetic surgical procedures to, for example, remove skin lesions andperform skin resurfacing.

Before describing the invention in detail, the following briefdescription of the various anatomical features of the eye is provided,which will help in the understanding of the various features of theinvention:

Referring to FIG. 1, the cornea 10, which is the transparent window thatcovers the front of the eye 100, is a lens-like structure that providestwo-thirds of the focusing power of the eye.

The cornea 10 is slightly oval, having an average diameter of about 12mm horizontally and 11 mm vertically. The central thickness of thecornea 10 is approximately 550 μm.

The sclera 16 is the white region of the eye, i.e. posterior five sixthsof the globe. It is the tough, avascular, outer fibrous layer of the eyethat forms a protective envelope. The sclera is mostly composed of densecollagen fibrils that are irregular in size and arrangement (as opposedto the cornea). The extraocular muscles insert into the sclera behindthe limbus.

The sclera 16 can be subdivided into 3 layers: the episclera, scleraproper and lamina fusca. The episclera is the most external layer. It isa loose connective tissue adjacent to the periorbital fat and is wellvascularized.

The sclera proper, also called tenon's capsule, is the layer that givesthe eye 100 its toughness. The sclera proper is avascular and composedof dense type I and III collagen.

The lamina fusca is the inner aspect of the sclera 16. It is locatedadjacent to the choroid and contains thin collagen fibers and pigmentcells.

The pars plana is a discrete area of the sclera 16. This area is avirtually concentric ring that is located between 2 mm and 4 mm awayfrom the cornea 10.

The vitreous humor or vitreous 12 is the largest chamber of the eye 100(i.e. ˜4.5 ml). The vitreous 12 is a viscous transparent gel composedmostly of water. Unlike the fluid contained in the frontal parts of theeye (e.g., aqueous humor, discussed below), which are continuouslyreplenished, the transparent gel in the vitreous chamber is stagnant.

As is well known in the art, the vitreous humor 12 also contains arandom network of thin collagen fibers, mucopolysaccharides andhyaluronic acid.

The aqueous humor 14 occupies the anterior chamber 18 of the eye 100.The aqueous humor 14 has a volume of about 0.6 mL and provides nutrientsto the cornea 10 and lens 28.

One of the most important functions of the aqueous humor 14 is tomaintain IOP by the rate of its production and drainage.

The additional parts of the eye that are illustrated in FIG. 1 comprisethe uvea, and structures thereof, lens 28 and retina 30.

The uvea refers to the pigmented layer of the eye 100 and is made up ofthree distinct structures: the iris 22, ciliary body, and choroid 24.The iris 22 is the annular skirt of tissue in the anterior chamber 18that functions as an aperture. The pupil is the central opening in theiris 22.

The ciliary body is the 6 mm portion of uvea between the iris 22 andchoroid 24. The ciliary body is attached to the sclera 16 at the scleralspur. It is composed of two zones: the anterior 2 mm pars plicata, whichcontains the ciliary muscle 26, vessels, and processes, and theposterior 4 mm pars plana.

The ciliary muscle 26 controls accommodation (focusing) of the lens 28,while the ciliary processes suspend the lens 28 (from small fibers, i.e.zonules) and produce the aqueous humor 14 (the fluid that fills theanterior and posterior chambers and maintains intraocular pressure).

The choroid 24 is the tissue disposed between the sclera 16 and retina30. The choroid 24 is attached to the sclera 16 at the optic nerve 20and scleral spur. This highly vascular tissue supplies nutrients to theretinal pigment epithelium (RPE) and outer retinal layers.

The layers of the choroid 24 (from inner to outer) include the Bruch'smembrane, choriocapillaris and stroma. Bruch's membrane separates theRPE from the choroid 24 and is a permeable layer composed of thebasement membrane of each, with collagen and elastic tissues in themiddle.

The crystalline lens 28, located between the posterior chamber and thevitreous cavity, separates the anterior and posterior segments of theeye 100. Zonular fibers suspend the lens from the ciliary body andenable the ciliary muscle to focus the lens 28 by changing its shape.

The retina 30 is the delicate transparent light sensing inner layer ofthe eye 100. The retina 30 faces the vitreous and consists of two basiclayers: the neural retina and retinal pigment epithelium. The neuralretina is the inner layer. The retinal pigment epithelium is the outerlayer that rests on Bruch's membrane and choroid 24.

As indicated above, conventional slit lamp systems, which are oftenemployed to treat various eye disorders, such as diabetic retinopathy,require a contact lens to neutralize the refractive power of the corneaand to provide a variable field of view of the retina.

The length of time to perform a surgical procedure with a conventionalslit lamp system is also presently in the range of 30 minutes to anhour. There is thus an increased percentage of probable error due to thelaser photo-coagulation being controlled manually, i.e. by thephysician's hand, and the potential eye movements from the patientduring this extended period of time.

The present invention substantially reduces or eliminates thedisadvantages and drawbacks associated with conventional slit lampsystems and associated methods. As discussed in detail herein, thelaser-imaging apparatus include means for taking and storing digitalimages of the target eye structure(s), which can be retrieved on amonitor for diagnosis and defining the area of treatment. In someembodiments, the laser-imaging apparatus (and systems) of the inventionthus include a retinal camera (e.g., Topcon, Zeiss, Kowa or preferably awide angle viewing system having an elliptical mirror), which, in someembodiments, is modified with a wide-angle lens.

A wide field scanning laser-imaging apparatus, such as the laseropthalmoscope disclosed in U.S. Pat. No. 5,815,242, can also be employedto provide images of the fundus, particularly, the retina. The notedlaser-imaging apparatus can also be readily modified for lasercoagulation procedures in one or multiple health care applications oradditional vision care offices.

According to the invention, the viewing light can comprise a white lightfrom a flush light, laser source or one or more scanning lasers withcompensatory wavelengths in the range of approximately 190 nm-10,000 nm,more preferably, in the range of approximately 400-1060 nm, to obtain afundus photograph.

According to the invention, the retinal camera is connected to lasertransmission means (or a laser system) that is adapted to generate andtransmit laser energy that is sufficient to coagulate any desiredportion of the retina using a monitor's touch screen.

In a preferred embodiment of the invention, the laser-imaging apparatusincorporates aiming and treatment laser generating and transmissionmeans into an existing camera (e.g., Topcon, Zeiss, etc.) or a wideangle viewing system/camera, such as the laser opthalmoscope disclosedin U.S. Pat. No. 5,815,242. The laser-imaging apparatus also includescontrol means for controlling the aiming and treatment laser means, andthe camera.

Preferably, the transmitted laser beam passes through the optical pathof the viewing apparatus (or system) and is preferably reflected off ofan elliptical mirror in the camera (in which the imaging light isfocused toward the pupil or slightly behind the pupil), providing afield of view greater than approximately 200°.

Referring now to FIG. 2, there is shown one embodiment of alaser-imaging apparatus of the invention. As illustrated in FIG. 2, thelaser-imaging apparatus 200 includes a Topcon digital camera 202,scanning laser visualization means 204 and internal laser generation andtransmission means (i.e. coagulation means) 206. The laser-imagingapparatus 200 further includes a refracting lens 203, at least onetwo-way mirror 208 and a plurality of appropriately positionedreflecting mirrors 201 a-201 c.

Referring now to FIG. 3, in an alternative embodiment, a wide anglecamera equipped with an elliptical mirror 220 is employed. According tothe invention, the illustrated wide angle camera provides an improvedrange of between approximately 1500 and approximately 200°, inclusive,(or an improved range between 1500 and 200°, inclusive) of the retinafor optimal treatment.

In some embodiments, the concave, elliptical mirror 220 illustrated inFIG. 3 is configured to oscillate (or wobble) slightly in order to shiftthe focal point of the mirror 220 slightly from one side of the pupil tothe other side, thereby permitting the scanning light (e.g., lowcoherent wavelengths, etc.) inside the eye to cover a larger peripheralfield than possible without oscillation. An exemplary imaging system forimaging the central and peripheral retina, which employs such anoscillating concave mirror, is disclosed in Applicant's U.S. Pat. No.8,070,289; which is incorporated by reference herein in its entirety.

In other embodiments, the concave, elliptical mirror 220 illustrated inFIG. 3 is stationary and is not configured to oscillate or wobble. It isalso to be understood that the concave mirror 220 can be provided in theform of circular mirror, as well as an elliptical mirror.

FIG. 3 shows the path of the viewing/imaging scanning laser beams 214 asthey are reflected and pass through the focal point of the mirror behindthe pupil (Black) toward the retina. As illustrated in FIG. 3, thecoagulative laser beam 216 preferably passes through the same path asthe viewing/imaging beams 214.

In some embodiments of the invention, the laser-imaging apparatus of theinvention also includes an optical coherence tomography (OCT) means.

Preferably, the rays reflected back from the retina pass through thesame pathway and form a digital image that can be observed on themonitor.

According to the invention, the coagulative laser beam is also scannedover the retinal area via the same light path as used for theobservation and documentation.

The integration of the aiming and treatment laser generating andtransmission means with a camera requires the introduction of precisionmotorized optical fixtures. An opto-mechanical system having an integralcontrol system is thus provided to control and/or position the targetspot of the laser beam(s) in x and y directions within the eye. Thesystem is designed and adapted to interface with joystick commandsand/or computer/monitor touch screen commands, for local and remotecontrol of the aiming and treatment (or coagulative) lasertransmissions, respectively.

In some embodiments of the invention, the control means for positioningthe laser transmissions (or beams) within the eye consists of two maincomponents. The first component is adapted to move the beams in thex-direction. The second component is adapted to move the beams in they-direction.

Preferably, movement in the y-direction is provided by a mirroredsurface disposed in the optical path of the camera. This y-direction,motorized fixture provides precise movement of the mirrored surface,while still allowing diagnostic and treatment images to be seen throughthe retinal camera.

In some embodiments of the invention, producing the x-direction movementinvolves physically moving the laser unit; the movement of the laserbeing either translational or rotational. Various conventional means ofmotorized movement can also be employed to provide movement in thex-direction.

In a preferred embodiment, the laser-imaging apparatus is incommunication with another remote system via the Internet®, whereby thelaser-imaging apparatus can be controlled by a physician at the remotesite (e.g., medical center).

According to the invention, location of laser energy or beam applicationcan be from 5-200° of the retina. In some embodiments, location of laserenergy application is preferably 30-2000 of the retina.

In some embodiments of the invention, the transmitted coagulative laserenergy (or beam(s)) has a wavelength in the range of approximately400-1650 nm, more preferably, in the range of approximately 530-670 nm.The laser (or laser energy) can also be transmitted in a pulsed manneror continuously.

According to the invention, the laser spot size can be in the range ofapproximately 10 micron-1500 micron.

According to the invention, exposure time of the laser energyapplication can be in the range of approximately 1 femto-seconds to 1000seconds.

In some embodiments of the invention, the laser-imaging apparatus 200includes a photoacoustic system that can measure the temperature insidethe eye tissue during and after laser scanning. A preferredphotoacoustic system is disclosed in Applicant's U.S. Pat. No.8,121,663; which is incorporated by reference herein in its entirety.

As set forth in detail in the '663 patent, the photoacoustic system isadapted to record the sonic waves that are generated by heating eyetissue, e.g. retina tissue. This provides precise information of thetemperature generated as a result of the laser transmission, i.e.coagulative laser energy.

Advantageously, the photoacoustic system enables the temperaturegenerated at the treatment site to be measured. As a result, the systemis capable of balancing the energy of the laser system so that thecoagulation is performed in a uniform fashion at the desired area,without such balancing one could have some lesions stronger than othersdepending on the degree of the pigmentation of the retina at theparticular site (i.e., if the site absorbs more laser light).

Referring now to FIG. 4, there is shown one embodiment of aphotoacoustic system 80. As illustrated in FIG. 4, the photoacousticsystem 80 includes a laser source 88, an ultrasonic detector 89, and aprobe module 82. The probe module 82 includes an objective lensstructure 84, which is preferably coupled to the light source 88 via afiber optic connection or other light transmitter. Alternatively, thelight source can be incorporated into the probe module 82.

According to the invention, the light source 88 can comprise a laser,laser diode or superluminescent diode (SLD), as appropriate forgenerating the desired light wavelength and intensity. The light canalso be delivered as pulses or as modulated radiation.

As further illustrated in FIG. 4, the probe module 82 further containsan ultrasound transducer 86 that is adapted to detect the photoacousticwaves that are generated as a result of the absorption of energy fromthe light emitted by the objective lens structure 84. The ultrasoundtransducer 86 is in contact with the eye 100 or an eyelid drawn over theeye.

As light is delivered as pulses or as modulated radiation, pulses ormodulating acoustic signals are generated and returned to the ultrasoundtransducer 86 in probe module 82.

According to the invention, localization of the source of photoacousticsignals can be achieved in various manners. First, localization can beaccomplished by directing the beam from objective lens structure 84 inspecific directions, by moving that structure with micromechanicalactuators, as shown diagrammatically at 85 in FIG. 4, thus targeting aparticular line of points in the eye.

Furthermore, by suitable optics included in objective lens structure 84,the focal point of the emitted light may be moved within the eye to adesired point, such as a point along the retina vasculature, toselectively generate acoustic signals at that desired point. Because theeye is optically transmissive relative to soft tissue, beam focusing andbeam directing are likely to be more accurately performed in the eye,than is the case is soft tissue elsewhere in the body.

To further assist in directionally capturing the photoacoustic signalsgenerated within the eye, a directional transducer array can be used astransducer 86, to control the directionality of reception of ultrasonicenergy, thus further localizing upon a desired source of thermoacousticsignals. Thus, by targeting the focal point of the illuminating light,and also directionally targeting the reception of ultrasonic signals bythe transducer array, thermoacoustic signals from a particular location,such as along the retina, may be specifically targeted.

Overview of Laser-Imaging System

Referring now to FIG. 5, there is shown a schematic diagram of alaser-imaging system 50, according to one embodiment of the invention.As illustrated in FIG. 5, the system 50 includes at least one,preferably, multiple local systems 52, 54, 56 disposed at local sites 51a, 51 b, 51 c; each system 52, 54, 56 including a laser-imagingapparatus (denoted “53”, “55” and “57”), such as the apparatus 200 shownin FIG. 2. In a preferred embodiment of the invention, eachlaser-imaging apparatus 53, 55, 57 includes a photoacoustic system, suchas system 80 discussed above.

Preferably, each laser-imaging apparatus 53, 55, 57 is preferably incommunication with a local control module 62 and control-processingmeans 59 a, such as a personal computer. Each of the modules describedherein in conjunction with the embodiments of the laser-imaging systems50, 50′ may comprise one or more hardware components together with therequisite software to carry out the functionality performed thereby.

In some embodiments of the invention, at least the control-processingmeans 59 a disposed at each local site 51 a, 51 b, 51 c includes facialrecognition means for identifying and/or verifying the identity of asubject or patient. Alternatively, in some embodiments, thecontrol-processing means 59 b disposed at the remote site 51 d(discussed below) includes facial recognition means. In some embodimentsboth control-processing means 59 a, 59 b include facial recognitionmeans.

In some embodiments of the invention, each local system 52, 54, 56 alsoincludes eye tracking means 71 for measuring eye position(s) andmovement. According to the invention, the eye tracking means 71 can bean integral component or feature of the laser-imaging apparatus 53, 55,57 or a separate system or device.

Also disposed at each local site 51 a, 51 b, 51 c during a laserprocedure is a test subject or patient and a physician or technician.

As also illustrated in FIG. 5, the system 50 also includes a remote site58 having a command computer 59 b that is operatively connected to aremote control module 64. Also disposed at the remote site 58 during alaser procedure is a system operator (e.g., retinal surgeon).

As discussed in detail below, communication by and between the localsites 52, 54, 56 and the remote site 58 is preferably facilitated by thelocal and remote control modules 62, 64 and the Internet® 60.

In accordance with one embodiment of the invention, the sequence ofinteractions between the local sites 52, 54, 56 and remote site 58comprises the following:

-   -   Fundus photograph is digitally transmitted to remote site;    -   Image is acquired by remote site and filed in the computer at        remote site;    -   Eye is repositioned in front of the camera, fundus image is        taken for pattern recognition and tracking, and transmitted to        remote site, verified as matching previous, pre-existing image        on file;    -   New image is selected and employed as simulator;    -   Spot size, power level, and time interval between each laser is        chosen;    -   Tracking system (fail-safe) is checked;    -   Fundus laser treatment is performed in virtual mode to establish        the desired laser coagulation; and    -   After choosing spot size and duration of laser application,        power is adjusted to the lowest increment of energy that may or        may not be able to create a response on the patient's eye. The        power of the laser is incrementally increased until the test        spot demonstrates the desired effect.

In a preferred embodiment, the optimum power and, hence, temperaturerequired for a procedure is provided via the photoacoustic system. Thephotoacoustic system is further adapted to maintain the transmittedlaser energy at a fixed, pre-selected level. This particularly resultsin a uniform coagulation of the retina.

Local and Remote Control Modules

Referring now to FIG. 6, there is shown a schematic diagram of a localcontrol module 62 and a remote control module 64, according to oneembodiment of the invention. As illustrated in FIG. 6, the local controlmodule 62 preferably includes three sub-modules: an operation module 65,safety and verification module 66, and an operation and performancesimulation module 67.

The remote control module 64 similarly includes three sub-modules: anoperation module 70, safety and verification module 72, and an operationand performance simulation module 74.

Each of these sub-modules 65, 66, 67 is described below.

Local Operation Module

According to the invention, the local operation module 65 provides alocal technician with an interface to a personal computer for dataacquisition and eye treatment.

According to the invention, the major tasks performed by the localoperation module 65 include the following:

(1) acquiring a plurality (preferably, in the range of 5-7) standardfields of the fundus retinal images and transmission of the images tothe remote site 58;

(2) receiving the oval area encompassing the focused region of theretina outlined by the remote physician (see FIGS. 5 and 7), as well asparameters for the spot size, power and duration of laser application;

(3) applying a snake algorithm (i.e. active contour algorithm) toextract the contours, calculating the corresponding areas between thecontours, and partitioning the image into a grid map (see FIG. 10), as afunction of the specified surgical area and corresponding parameters forlaser application from the remote center. According to the invention,the resolution of the grid is adjusted according to the area between thecontours, whereby the number of nodes between the contours (i.e. theblack points between the two white contours) is large enough to generatein the range of approximately 700-1000 laser spots to ensure the surgeryprecision;

(4) performing the scatter laser coagulation under the remote doctor'scommand. During the process (which is performed in real-time), theKalman filter and mean shift algorithm are preferably integrated todetect and estimate the test subject's eye movement: a) if the movementis under the given safety threshold, the laser system will be adjustedusing a simple proportional-integral-derivative (PID) control algorithmbased on the estimated motion so that it will be focused on the remotelyspecified spot within allowable surgical accuracy, and then the laserwill be applied; b) if the estimated movement is beyond the specifiedsafety threshold, the laser coagulation procedure will be terminatedimmediately and a warning message will be transmitted to the remotecontrol module 64. Step (1), above, will also be repeated until a stopinstruction is received from the remote physician; and

(5) acquiring a plurality (preferably, in the range of 4-5) standardfields of fundus retinal images and transmitting the images to theremote site for evaluation and verification of treatment.

Local Operation and Performance Simulation Module

According to the invention, the local operation and performancesimulation module 67 allows the physician (or technician) to test theentire system 50 (e.g., tracking and interruption functions in the localoperation module 65; communications between local and remote sites;safety and verification modules 66, 72 at both remote and local sites)before the system 50 is run in an actual control mode.

In the simulation mode, the local simulation module 67 replaces the testsubject (or patient), but all the other modules (e.g., modules at theremote site 70, 72, 74; safety and verification module 66 and localoperation module 65 at the local site) preferably operate in exactly thesame manner.

In one or more embodiments of the invention, the local operation andperformance simulation module 67 is configured to test the trackingfunctions of the local operation module 65 by replacing the test subjectwith a digitized fundus image of him or her. The tracking system istested using the digitized fundus image in place of the test subject bydisplacing the digitized fundus image in the X, Y, and O directions (seeFIG. 9). The displacement of the digitized fundus image simulates thehead and/or eye movement of the test subject that is experienced in theactual control mode. The laser is activated in the simulation mode, butis configured to turn off if the movement of the digitized fundus imageexceeds a certain predetermined threshold so as to simulate the actualmovement of the test subject exceeding a predetermined threshold.

Any rapid displacement beyond a certain predetermined threshold value orrange of values (e.g., an orientation change exceeding 3-5 degrees)cannot be immediately compensated for by the tracking system. As aresult, in such a situation, the local operation and performancesimulation module 67 is configured to simulate the response of thetracking system to the detection of a displacement exceeding thethreshold value or range by shutting down the laser of the lasercoagulation system. In the simulation mode, the digitized fundus imagecan be slightly tilted or laterally displaced (e.g., moving the image onthe screen of a visual display device or mechanically, by displacing thescreen itself containing the image with a multi-dimensional actuator) tosimulate the deactivation of the laser. In some embodiments, thetracking system is configured to follow a particular spot on the fundusimage. When the spot being followed by the tracking system is rapidlytilted or displaced, the laser is shut down by the laser coagulationsystem. This simulated test is used to ensure that the tracking systemis fully operational and functioning properly prior to the performanceof the laser surgery (i.e., in the actual control mode).

While performing the simulation using a digitized fundus image of theeye is preferred, a physical model of an eye can also be used to performthe simulation carried out by the local operation and performancesimulation module 67. For example, an artificial eye can be placed infront of the lens of the laser-imaging apparatus. In such an artificialeye, the retina of the eye is visible through the pupil thereof. In thisconfiguration, the tracking system of the laser coagulation system istested by slightly displacing the artificial eye in front of the lens ofthe laser-imaging apparatus (e.g., by using mechanical means, such as amulti-dimensional mechanical actuator) while the laser is activated, butis not firing any actual laser shots. When the detected displacementexceeds the threshold value or range, the tracking system is triggeredso as to shut down the laser.

Remote Operations Module

According to the invention, the remote operation module 70 provides thephysician with an interface to a personal computer. During a laserprocedure, some of the important tasks that the physician will performand are facilitated by the remote operation module) include: (1)screening via digitized photos, (2) outlining an oval area encompassingthe center part of the retina (see FIGS. 5 and 7), (3) choosing the spotsize and duration of laser application, (4) adjusting the power viasingle shots of test spots (preferably, with a duration in the range ofapproximately 0.001-0.1 seconds in approximately 0.06 intervals), (5)executing virtual treatment with the simulation module (see FIG. 6), (6)performing a test surgery, which involves the local operation andperformance simulation module 67, and (7) performing the laser surgeryin the actual control mode (and observing the surgery via a real-timevideo stream).

In one or more embodiments of the invention, the remote operation module70 further comprises an electronic visual display device withtouchscreen capabilities, which is operatively coupled to a personalcomputer or another digital appliance that has processing capabilities(e.g., a portable digital device, such as a mobile phone or smartphone,a laptop computing device, a palmtop computing device, a tabletcomputing device, etc.) As such, the laser-imaging apparatus (i.e., thefundus camera) is operatively coupled to the visual display device ofthe remote operation module 70 so that the digitized fundus (retinal)image of the patient is able to be displayed in detail.

The touchscreen system of the remote operation module 70 comprises adisplay screen that is sensitive to the touch of a finger or a type ofstylus (e.g., a pen stylus). The touchscreen-type visual display deviceincludes: (i) touch sensor in the form of a panel with a responsivesurface, (ii) a hardware-based controller, and (iii) touchscreensoftware executed by the personal computer or other digital appliance.The touch sensor employed by the touchscreen may comprise resistive-typesensor(s), capacitive-type sensor(s), or surface acoustic wavesensor(s). Each of these sensor types has an electrical current passingthrough them, and touching the surface of the screen results in aconsequential voltage change. The voltage change is indicative of thelocation of the touching. The controller is the hardware component thatconverts the voltage changes produced by the touch sensor into signalsthat the personal computer or other digital appliance can receive. Thetouchscreen software instructs the personal computer or other digitalappliance as to what is occurring on the touchscreen and on theinformation delivered from the controller (i.e., what the user istouching and the location of his or her touch) so that the computer ordigital appliance can respond accordingly.

Using the touchscreen visual display device, the physician (i.e., theophthalmologist) can control the laser system with the touch of his orher finger or by using a stylus pen, etc. The visual display device alsoincludes zoom capabilities in order to allow the physician to examinethe details of the digitized fundus image (i.e., retinal image) ifneeded. The display recognizes and interprets the commands of thephysician and communicates those commands to the personal computer thatcontrols the laser generation system. As such, the laser generationsystem is controlled in accordance with the markings made by thephysician on the touchscreen. In other words, the laser generationsystem is configured to carry out the laser treatment in accordance withthe indicated markings on the touchscreen. When a typical procedure isbeing carried out by the physician, the digitized fundus (retinal) imageof the patient is initially recalled on the visual display device.Because the fundus image is digitized, the physician is able toprecisely sketch/draw on any area of the digitized fundus image so thathe or she can indicate the location(s) where the laser pulses should beapplied, and as required, the location(s) where the laser pulses shouldnot be applied. For example, the area or areas where the laser pulsesare to be applied are shaded and/or denoted using a first predeterminedcolor (e.g., green, which signifies a laser is to be applied thereto),while the area or areas where laser pulses are not to be applied areshaded and/or denoted using a second predetermined color (e.g., red,which signifies that a laser is not to be applied in this region orregions). When the laser coagulation system is operating in either thesimulation mode or the actual control mode, any invasion of the areashaded with the second predetermined color (i.e., the red area) willresult in the immediate shutting down of the laser. Also, the area(s)where the laser pulses are to be applied can be separated from the restof the retina by a continuous line of a predetermined color (e.g., a redline) so that there is a clear demarcation on the retinal image. Thearea or areas that are shaded and/or denoted using the secondpredetermined color (i.e., red, indicating that laser pulses are not tobe applied thereto) may represent a part of the retina containing largeretinal vessels, such as the region with the optic nerve head and thefovea, which are sensitive to laser damage and should not be coagulated(e.g., see FIG. 11). The area of the laser coagulation can be contiguousor separated. In general, the distance between each of the laser pulsescan vary (e.g., in the range between 0.1 mm and 4.0 mm or betweenapproximately 0.1 mm and approximately 4.0 mm, inclusive). The laserapplication can be a single pulse as long as it is indicated by a singlecolored dot (e.g., a green dot), or the quantity of the laser spots canrange up to 2500 spots or more. The laser pulse duration can be in therange from one femtosecond to 4.0 seconds, inclusive (or in the rangefrom approximately one femtosecond to approximately 4.0 seconds,inclusive). In other embodiments, the laser pulse duration can exceed4.0 seconds. The energy level of each pulse can range from 0.01femtojoule to one joule, inclusive (or between approximately 0.01femtojoule to approximately one joule, inclusive), and more preferably,between 1 nanojoule to one joule, inclusive (or between approximately 1nanojoule to approximately one joule, inclusive). The spot size of thelaser varies between 0.0001 nanometers (nm) to 2 millimeters (mm),inclusive (or between approximately 0.0001 nm to approximately 2 mm,inclusive).

In addition, in one or more embodiments, the visual display device withtouchscreen capabilities displays the degree of energy (e.g., number oflaser pulses, laser spot size, laser power or laser energy level) andthe number of laser spots applied for a given area. The visual displaydevice also displays all the parameters that are needed or chosen by theoperator in addition to the presentation of the virtual image of thetest laser application. All of this information is recorded and storedin the computer. The remote laser apparatus, which is located at each ofthe local sites 51 a, 51 b, 51 c (see FIG. 5), does not perform thelaser coagulation if the prior test is not performed. Similarly, afterthe correction of any parameter of the laser, or any change in the areaof application of the laser, a new test application is run to ensure thesafety of the procedure.

Also, as a part of the fail safe mechanism of the system, any invasionof the area which is not be subjected to the laser (e.g., the areacolored in red by the physician) results in the immediate shutting downof the laser so that the appropriate inspection can be performed and/orcorrective actions can be taken. As such, the system has completecontrol over the area of laser application.

Referring now to FIGS. 7 and 8, there are shown photographs of theretina, showing the outline of the oval target area (area bound by twooval lines) thereon for application of laser spots (FIG. 7) and thelaser spots (in some embodiments, in the range of approximately 50-3000laser spots) on the retina achieved via virtual treatment (see FIG. 8),as described below. In some embodiments, the physician can draw the ovallines in FIGS. 7 and 8 on the digitized fundus image by utilizing thetouchscreen-type visual display device.

Now, turning to FIGS. 11-14, four (4) images showing exemplary areas oflaser application on a patient are illustrated (e.g., as marked by aphysician using the touchscreen-type visual display device). FIGS. 11-13depict laser application areas in the retina, which have been indicatedon the fundus (retinal) image of a patient, while FIG. 14 depicts alaser application area on the skin surface of a patient. Specifically,in FIG. 11, it can be seen that the annular area 300 (or donut-shapedarea 300) with diagrammatic arrows 301 has been marked for laserapplication (e.g., by using a green color), while the central circulararea 302 has been marked for no laser application thereto (e.g., byusing a red color). As described above, the fovea 304 and the opticnerve head 306, which are located in the central circular area 302, arehighly sensitive to laser damage, and thus, should not be coagulated. Asshown in FIG. 11, the retinal area has a plurality of blood vessels 308disposed throughout; the retinal area is surrounded by the or a serrata310 (i.e., the serrated junction between the retina and the ciliarybody). Next, turning to FIG. 12, two (2) localized areas 312, 314 aremarked for laser application (e.g., by using a green color). As shown inFIG. 12, the localized areas 312, 314 are generally elongated, curvedregions of the retina. Then, with reference to FIG. 13, it can be seenthat a plurality of single localized spots 316 on the retina are markedfor laser application (e.g., by using a green color). As illustrated inFIG. 13, the plurality of localized spots 316 are disposed in closeproximity to the fovea 304 and the optic nerve head 306.

In addition, it is to be understood that the laser coagulation proceduredescribed herein can also be applied to external surface areas on thebody of a patient. For example, in FIG. 14, an area 320 of skin lesion319 on a patient's face 318 is marked for laser application (e.g., byusing a green color). As other examples, the laser coagulation systemdescribed herein could be used to treat the eye surface, the cornea,conjunctiva, eyelid, skin areas, or visible mucosa. As such, any part ofthe eye can be treated with the laser coagulation system describedherein using the same laser-imaging apparatus described above, or amodified camera technology that is particularly suited for surfaceapplications (e.g., a standard camera, such as those that are used inphotography, or a microscope, etc.). Such a camera would be used tocreate images of the eye surface, the cornea, conjunctiva, eyelid, skinareas, or visible mucosa, etc.

Remote Operation and Performance Simulation Module

According to the invention, the remote operation and performancesimulation module 74 allows a physician or technician to perform virtualtreatment, which permits the physician to test the local operationmodule 65 (at a local site) in terms of its generation capability of thelaser spots throughout the surgery area specified by the doctor (seeFIG. 8). The algorithm used in this simulation module is similar to theone used by the local operation module 65.

After the physician (i.e., ophthalmologist) outlines the extent of thedesired treatment area using the touchscreen visual display device ofthe remote operation module 70, he or she initiates the simulationprocess using the remote operation and performance simulation module 74.The treatment area outlined by the physician is filled virtually by themodule 74 with dots (e.g., white dots), thereby covering the area andindicating the extent and the density of the laser spots that shouldoccur on the retina and not on any other area.

Referring now to FIG. 9, in some embodiments, movement of the object(i.e. digitized fundus photograph) is simulated via three randomvariables; X, Y and O. The variables X and Y denote the displacementchanges in the X and Y axes, respectively. The variable O denotes theorientation changes. In a preferred embodiment, the digitized fundusimage is displaced electronically by the remote operation andperformance simulation module 74 using the variables X, Y and O. Thedisplacement of the digitized fundus image represents the movement ofthe retinal image of the patient prior to operating the lasercoagulation system in the actual control mode.

In another embodiment, it is also possible to utilize mechanical meansto displace a hardcopy of the fundus image (e.g., by using amulti-dimensional mechanical actuator that is operatively coupled to thehardcopy of the fundus photograph).

To this end, movement data from test subjects is first collected, fromwhich best-fit statistical distributions (and variances) for the threerandom variables (X, Y, and O) are determined. According to theinvention, a “goodness-of-fit” test can then be performed to test thevalidity of the distribution.

If a theoretical distribution (involving larger p-value in statisticalsense) exists, it will then be employed. Otherwise, an empiricaldistribution will be constructed and employed.

According to the invention, the remote operation and performancesimulation module 74 also allows the physician to test the localoperation module 67 (at a local site) in terms of its generationcapability of the laser spots without involving communications with thelocal control module 62.

Safety and Verification Modules

According to the invention, safety and verification modules 66, 72 existboth at the remote site 58 as well as each local site 52, 54, 56 toensure safe and effective operation of the system 50. In a preferredembodiment of the invention, several constraints are preferably imposedinto the system 50 (both hardware and software). In some embodiments,the constraints include (1) physical constraints, (2) logicalconstraints, and (3) medical constraints.

According to the invention, physical constraints ensure variousparameters or values (e.g., therapy beam power) in the laser system 50are within a permitted range. If any of the values are outside apermitted range, laser firing will be automatically locked andnotification of the unacceptable value(s) is transmitted to thephysician at the remote site, as well as the technician at the localsite.

Logical constraints are employed to ensure that a correct sequence ofoperational tasks is performed. For example, if a physician at a remotesite mistakenly commands 700-1000 laser spots of laser application tothe fundus before simulating the laser application in a simulation mode,the system will not execute the command and transmits a warning messageto the physician. In a preferred embodiment, Unified Modeling Language(UML) is incorporated into the system software and employed to specifythe logical constraints.

The medical constraints involve monitoring of the fundus of the testsubject(s) during the actual laser procedure operation. If it isdetected that the laser energy effect on the fundus is different fromwhat the physician expected or the laser energy is applied beyond thespecified area, laser energy transmission is immediately terminated.Notification of the issue is also transmitted to the physician at theremote site, as well as the physician or technician at the local site.

As indicated above, in some embodiments of the invention, the system 50also includes eye tracking and facial recognition means to ensure safeoperation.

Laser-Imaging System with Image and Voice Recognition Capabilities

Referring now to FIG. 15, there is shown a schematic diagram of alaser-imaging system 50′, according to another embodiment of theinvention. Referring to this figure, it can be seen that, in manyrespects, this embodiment of the laser-imaging system 50′ is similar tothe embodiment of the laser-imaging system 50 described above. Moreover,many elements are common to both such embodiments. For the sake ofbrevity, the elements that the laser-imaging system 50′ has in commonwith the laser-imaging system 50 will only be briefly mentioned, if atall, because these components have already been explained in detailabove. Furthermore, in the interest of clarity, these elements aredenoted using the same reference characters that were used in thepreceding embodiment.

As illustrated in FIG. 15, like the system 50 described above, thesystem 50′ includes a plurality of local control systems disposed atlocal sites 51 a′, 51 b′, 51 c′; each system including a laser-imagingapparatus 55, such as the apparatus 200 shown in FIG. 2. In a preferredembodiment of the invention, each laser-imaging apparatus 55 includes aphotoacoustic system, such as system 80 discussed above.

Preferably, each laser-imaging apparatus 55 of the system 50′ ispreferably in communication with a local control module 62 andcontrol-processing means 59 a. In the illustrated embodiment, thecontrol-processing means 59 a may be embodied as a local personalcomputer or local computing device that is specially programmed to carryout all of the functionality that is described herein in conjunctionwith the local control systems of the laser-imaging system 50′.

In the illustrated embodiment of FIG. 15, each local control system alsoincludes eye tracking means 71 for measuring eye position(s) andmovement. According to the invention, the eye tracking means 71 can bean integral component or feature of the laser-imaging apparatus 55 or aseparate system or device.

Referring again to FIG. 15, it can be seen that each local controlsystem also includes an image recognition sensor 73 for capturing imagesof a subject or patient that may be used to identify and/or verify theidentity of the subject or patient. Advantageously, the positiveidentification and verification of the identity of the subject orpatient receiving treatment prevents surgical mistakes wherein the wrongsubject or patient is treated. In addition, rather than identifyingand/or verifying the identity of the subject or patient, the imagerecognition capabilities of the image recognition sensor 73 may also beused to identify and verify that a particular surgical procedure isbeing performed on the proper body portion of a subject or patient(e.g., to verify that a laser coagulation procedure is being performedon the proper one of the patient's eye or that a surgical procedure isbeing performed on the proper one of the patient's limbs). As will bedescribed hereinafter, in a further embodiment, the image recognitionsensor 73 or imaging device may be provided as part of a dynamic imagingsystem that is operatively coupled to the laser-imaging apparatus 55 ofthe laser treatment system. Also, as will be described hereinafter, theimage recognition sensor 73 or imaging device may comprise a light fieldcamera that is able to simultaneously record a two-dimensional image orthree-dimensional image and metrically calibrated three-dimensionaldepth information of a scene in a single shot. In one or moreembodiments, the image recognition sensor 73 may also comprise amultispectral camera that captures multispectral digital images over awide range of frequencies of the electromagnetic spectrum (e.g., fromvisible light frequencies to infrared radiation frequencies). As such,because the camera is capable of operating using a wide range ofdifferent frequencies, the multispectral camera may be used as both avisible light camera (e.g., operating in the 450-750 nanometer range)and a thermographic camera for capturing images using infrared radiation(e.g., operating with wavelengths up to 14,000 nm (14 μm)).

In one or more embodiments, the image recognition sensor 73 of eachlocal control system may be operatively connected to the local computingdevice, which forms the control-processing means 59 a of the localcontrol system. The local computing device may be specially programmedwith image/pattern recognition software loaded thereon, and executedthereby for performing all of the functionality necessary to identifyand verify a particular subject or patient, or to identify and verify abody portion of the particular subject or patient that is to be treated.Initially, the local computing device may be specially programmed tocapture and store a first reference digital image of a body portion ofthe subject or patient so that the first reference digital image may becompared to a second digital image of the same body portion captured ata later time (i.e., just prior to the performance of the surgicalprocedure). Then, prior to the performance of the surgical procedure(e.g., a laser coagulation performed on the eye), the second digitalimage of the same body portion of the subject or patient is captured bythe image sensor 73 (i.e., the multispectral camera) and the localcomputing device compares the second digital image of the body portionto the first reference digital image and determines if the seconddigital image of the body portion matches or substantially matches thefirst reference digital image. When the local computing devicedetermines that the second digital image of the body portion of thesubject or patient matches or substantially matches the first referencedigital image, the local computing device is specially programmed togenerate a matched image confirmation notification that is sent to theremote computing device at the remote site in order to inform theattending physician that the proper patient or body portion of thepatient has been identified and verified. The matched image confirmationnotification may also be delivered to the technician at the local sitevia the local computing device. Then, after the other safety checks ofthe system 50′ have been performed, the surgical procedure is capable ofbeing performed on the patient. Conversely, when the local computingdevice determines that the second digital image of the body portion ofthe subject or patient does not match or substantially match the firstreference digital image, the local computing device is speciallyprogrammed to generate a non-matching image notification that is sent tothe remote computing device at the remote site in order to inform theattending physician that the patient or body portion of the patient hasnot been properly identified and verified. The non-matching imagenotification may also be delivered to the technician at the local sitevia the local computing device. When the non-matching image notificationis sent to the attending physician, the local computing device alsodisables the surgical equipment at the local site in order to preventthe procedure from being performed on the incorrect patient or theincorrect body portion of the patient (e.g., in a laser coagulationprocedure, laser firing will be automatically locked out by the localcomputing device).

When the multispectral camera comprising the image recognition sensor 73is used as a thermographic camera in the mid-infrared or infraredradiation wavelength range, the camera may be used to capture images ofcapillaries in the skin of the patient or subject. As such, the cameramay sense the heat generated by the blood in these capillaries toidentify the subject or patient, or a particular body portion of thesubject or patient. A multispectral camera that is capable of bothvisible light photography and infrared pattern recognition takes intoaccount psychological functions which are not achievable with a camerahaving only visible light capabilities (e.g., two or three differentphysiological functions of the patient may be taken into account with amultispectral camera). For example, prior to capturing the infraredimage of a face of the subject or patient, the subject or patient may beinstructed to frown so that wrinkles are made prominent in the face ofthe subject or patient. When the subject or patient frowns, skincapillaries become collapsed and folded, thereby reducing the blood flowthrough the collapsed and folded capillaries and the heat that isdetected by the multispectral camera. As such, the local computingdevice may be specially programmed in the manner described above toverify the identity of the subject or patient using the image patterncreated by this particular facial expression (i.e., frowning).Detectable wrinkles may also be created by instructing the patient tolift up his or her brow, by instructing the patient to laugh, or byinstructing the patient to execute any other facial expression thatcreates wrinkles. As another example, the multispectral camera may beused to capture a body portion of the subject or patient (e.g., aportion of the patient's skin) that has previously undergone a surgicalprocedure whereby the structure of the body portion has been altered(e.g., the capillaries in a portion of the patient's skin that has beenpreviously operated on will have a very unique pattern that is specificto the particular patient being verified). The scars created by thesurgery create a very different, unique marker on the patient (e.g.,scar tissue created by surgery does not have many capillaries). Becausethis specific pattern of skin capillaries that were surgically alteredis extremely hard to artificially replicate, this means of verifying apatient's identity is very reliable and imposters are highly unlikely.

Now, turning to FIG. 16, exemplary image capture areas on a face 402 ofa patient 400 are shown. These are merely illustrative examples of someareas that may be imaged by the image recognition sensor 73 of the localcontrol system. The first image capture area 404 in FIG. 16 is disposedaround the left eye of the patient 400, and includes the left eye plusthe area surrounding the left eye (e.g., so that facial features, suchas wrinkles around the eye, may be used for patient identification andverification purposes in addition to the eye itself). The second imagecapture area 406 in FIG. 16 is disposed around the left corner of themouth of the patient 400, and includes a portion of the mouth and thearea proximate to the mouth (e.g., so that facial features, such aswrinkles around the mouth, may be used for patient identification andverification purposes). The third image capture area 408 in FIG. 16 isdisposed around the entire face 402 of the patient 400 (e.g., so that aplurality of different facial features, such as wrinkles disposedthroughout the face, may be used for patient identification andverification purposes). When the image of the face 402 of the patient400 is captured by the image recognition sensor 73, the wrinkles in theface 402 remain dark. It is very difficult to circumvent the imagerecognition system by replicating wrinkles. The fourth image capturearea 410 in FIG. 16 is disposed around the right eye of the patient 400(e.g., so that the various structures of the right eye, such as the irisand retina, may be used for patient identification and verificationpurposes).

In addition to the face 402 of the patient 400, it is to be understoodthat other body surfaces may be imaged by the image recognition sensor73 of the local control system as well. For example, any portion of thepatient's body where the physician would like to apply a laser lesionunder observation may be imaged by the image recognition sensor 73.Laser applications may vary from a nearly invisible laser lesionresponse from the tissue to an obvious burn of a given size or fortissue ablation and evaporation using any suitable laser, such as a gaslaser, a solid-state laser, an ultraviolet laser, an infrared laser(i.e., a mid-infrared laser up to a CO₂ laser for surface tumor removalor for rejuvenation for cosmetic surgery), an excimer laser (i.e., forcorneal surgery) with wavelengths of approximately 192 nm, and a CO₂laser with wavelengths of approximately 10,000 nm (10 micron). Othersuitable excimer lasers with various wavelengths may also be used (i.e.,excimer lasers with other ultraviolet wavelengths). In addition,microwave or radiofrequency radiation may be applied to the patient froman appropriate unit to achieve the desired therapeutic result. The beamdiameter may be varied or be focused to a diameter of one micron to fivemillimeters or more. The duration of each pulse can vary from onefemtosecond to a continuous laser application depending on the need ofthe operator to achieve a particular result.

In a further embodiment, the laser-imaging system 50′ may comprise animage recognition sensor 73 in the form of a holoscopicthree-dimensional (3D) camera at the local site (i.e., to capture 3Dimages of the patient), and the graphical user interface of the commandcomputer 59 b at the remote site may be in the form of a multiview,three-dimensional (3D) display (i.e., so the physician can view the 3Dimages of the patient for identification purposes). In this furtherembodiment, changes in the physiological condition of the patient areinduced (e.g., such as by having the patient smile, frown, etc.) whilethe changes are recorded by the 3D camera before, during, and after amedical procedure. Then, the results are displayed to the physician atthe remote site using the multiview 3D display. Advantageously, themultiview 3D display permits a three-dimensional analysis of thephysiological changes of the patient by the physician before and after aphysiological change is induced. Also, the physician is able to analyzethe trends of the physiological changes of the patient, such asaugmentation or diminution of the physiological change. Also, thephysical parameters associated with the physiological changes are ableto be analyzed using a computer at the remote site (i.e., computer 59b), and compared with the initial existing data.

In this further embodiment, changes in the physiological condition maybe induced in the patient after the patient responds to a question posedto him or her. In this case, the surface changes in the patent's facewhile answering the question may be recorded by the image recognitionsystem and analyzed for the patient's recognition by comparing therecorded results to a previous session. Also, the patent's speech may berecorded while he or she is answering the question so that the speechpatterns of the patient may be analyzed using the voice recognitionsensor 75 described hereinafter.

Also, in this further embodiment, the multiview, three-dimensional (3D)display of the image recognition system may comprise a digitalholographic display so as to allow the physician to perform athree-dimensional analysis of the changes in the physiological conditionof the patient which are induced (e.g., such as by having the patientsmile, frown, etc.). In this further embodiment, the digital holographicdisplay may permit 3D display while maintaining the phase and amplitudedata to reconstruct the wavefront through the use of a stereoscopic headmount display or console to reduce the sensory conflict of the eyeduring the convergence and accommodation and aid in a virtual realitydisplay of the head, face, extremities, retina, vessels etc., for threedimensional analysis of the physiological changes before and after thephysiological change is induced and the trends of the physiologicalchanges. Also, the 3D display may be in the form of virtual realityglasses that are worn by the treating physician at the remote site.

Moreover, in this further embodiment, the digital holographic display ofthe image recognition system may comprise one or more thin holographicoptical elements or one or more ultrathin optical elements for producinghigh-resolution 3D images, and may further comprise a multiviewautostereoscopic 3D display to eliminate the need for the physician towear special eye wear while performing the three-dimensional analysis ofthe induced changes in the physiological condition of the patient.

Furthermore, in this further embodiment, the multiview,three-dimensional (3D) display of the image recognition system maycomprise a volumetric 3D display so as to present a 3D image on whicheach voxel is appropriately positioned, and reflects light to form areal image for the viewer (i.e., the physician) providing physiologicaland perceptual hints and spatial depth, and volume with high resolutionof a structure while performing the three-dimensional analysis of theinduced changes in the physiological condition of the patient.

Additionally, in this further embodiment, the holoscopicthree-dimensional (3D) camera of the image recognition system may beconfigured to record tomographic images from a surface structure of thepatient, such as the skin, as deep as the light can penetrate the tissuefor three-dimensional analysis of the induced physiological changes andtheir trends. Also, the induced physiological changes of the patient maybe additionally recorded during and after imaging using a near-infraredlaser with Optical Coherence Tomography (OCT) or Near-Infrared opticalTomography (NIROT). When OCT is used for recording the changes,wide-angle Optical Coherence Tomography (OCT) may be utilized so thatchanges, such as the elevation or depression of a structure (e.g., thecheek, nose, tumor of the skin, etc.), is capable of being used during atwo or three-dimensional analysis of the physiological changes and theirtrends. In addition, the induced physiological changes of the patientalso may be recorded using a Frequency Modulated Continuous Wave (FMCW)system with ranging detection so as to obtain before, during, and afterthe physiological change is induced, 3D images of a surface whichdiffuses light with precision, i.e., under 10 micron speckle noise,while permitting the analysis of a variety of reflecting materials andtissue.

In another embodiment, the dynamic identity imaging involves analysis ofa person's physical or psychological changes by presenting to a subject,a range of images and/or sounds or videos of incidences of relaxation orpleasure to extreme sadness or pain, while recording the person's bodyor the face with a hyperspectral camera, etc. to observe his or herresponse to the images/sound that can be analyzed in a dynamic formatfor physical or physiological, medical, psychology, psychiatric, orinvestigational, medical diagnosis, or security field, etc. orevaluation of a symptom of a disease before and after therapy, or toobserve children in the playground, etc.

In yet a further embodiment, the image recognition sensor 73 of thelaser-imaging system 50′ may comprise two spaced-apart cameras thatcapture two standard images separated by 60 to 70 millimeters (mm) fromeach other (e.g., like two eyes of human, but as small as the camera ona smartphone). The two small cameras may be mounted over thephotographic and imaging system of the laser-imaging system 50′. Twoseparate, but simultaneous images are taken with a white flash lightgenerating a full spectrum of color (ultraviolet to infrared orinfrared, etc.) using a diode laser. The patient can perform a similaraction as that described above, such as smiling or frowning, which isthen recorded by the two small cameras. The image is transmittedseparately, but simultaneously via the Internet to the computer at theremote site (i.e., computer 59 b), which may be located at thephysician's or observer's office. This information is transmitted to areceiver and a small processor worn by the physician or observer, andconverted to two different light pulses via a fiber optic to a smallprismatic mirror located in front of each pupil of physician orobserver, and projected into the eyes of the doctor's or observer'sretina so that the right eye sees the images from the right camera andthe left eye sees the images from the left camera. Both cameras may beworking as a video system so that the doctor or observer sees a 3D videoof the patient's face from the time that the patient sits in front ofthe cameras up to the time that the treatment laser of the laser-imagingsystem 50′ is applied to the patient's face or to another part of thepatient's body. Using the camera(s), it is also possible to create a 3Dimage of the retina or an object, such as face, located close to thecamera(s). Also, in this further embodiment, an electronicallycontrolled rapidly moving prism (e.g., oscillating at greater than 60Hertz) or an electronically controlled rotatable lens (e.g., rotatingback and forth by 10 to 30 degrees) in front of a single camera may beprovided so as to create two different, but slightly angulated imagesfrom an outside object (e.g., face etc.) or from inside the eye of apatient. These separate images again are sent via the Internet to botheyes of the physician or an observer separately, but with the samefrequency (greater than 60 Hertz). These images are capable of creatingthe sensation of a 3D display for the physician or observer who sees twodifferent images with both eyes, but because of the frequency of theimage presentation, the images are fused in the brain and seen as onethree-dimensional image. In addition, the computer at the remote site(i.e., computer 59 b) may also be configured to separately analyze eachframe of the video so as to analyze changes that have occurred in thephysiological condition of the patient by virtue of the induced changes(e.g., smiling, frowning, etc.) so that the patient's identity may beverified before any treatment is initiated.

In still a further embodiment, the image recognition system of thelaser-imaging system 50′ may comprise a three-dimensional (3D)multi-color meta-holography device for recording, analyzing andtransmitting the data obtained during patient observation as a result ofchanges during and after a physiological action of the patent so as todistinguish newly computer-generated data from the previously storeddata. This embodiment utilizes metasurface, meta-hologram imaging and 3Dmeta-holographic imaging that has been made possible by advancement innanotechnology. In this embodiment, increased data capacity is achievedby the incorporation of single or different-sized plasmonic pixels intoa metasurface, so as to create 3D multicolor meta-holography withreduced overlap between different colors. In this embodiment, the imagerecognition sensor 73 of the laser-imaging system 50′ is in the form ofa 3D multicolor meta-holographic imaging device and, in case of adiscrepancy, the image recognition system is configured to shut down thelaser treatment system automatically while the doctor or observerlocated in at the remote site simultaneously views the 3D multicolormeta-holographic images produced by the image recognition system.

In yet a further embodiment, the photoacoustic system 80 described abovemay be used for ultrasonic three-dimensional (3D) imaging of bodystructures beneath the skin of the patient, such as bone structures inthe body of the patient. For example, the transducer array of thephotoacoustic system 80 may be used for obtaining an ultrasonic 3D imageof the bone structure of the nose of the patient while the head ofpatient is supported on a head support structure disposed in front ofthe photoacoustic system 80. Because the nose bone structure is uniqueto each patient, the imaged bone structure of the nose also may be foridentification purposes in the manner described above for skinstructures.

Turning again to FIG. 15, it can be seen that each local control systemmay further include a voice recognition sensor 75 for capturing thespeech waveforms generated by the subject or patient so that the speechof the subject or patient may additionally be used to identify and/orverify the identity of the subject or patient. In one or moreembodiments, the voice recognition sensor 75 may be used in conjunctionwith the image recognition sensor 73 described above to further verifythat a surgical procedure is being performed on the correct subject orpatient. In one or more embodiments, the voice recognition sensor 75 maycomprise a microphone that captures the speech of the subject or patientover the entire speech frequency range of a human being (e.g., for afrequency range from 50 Hz to 5,000 Hz to encompass the typicalfrequency range for both males and females). As such, the syntax andsound pulses generated by the subject or patient are capable of beingused by the local control system for verification and identification ofthe subject or patient prior to the surgical procedure being performedon him or her. In one or more embodiments, the voice recognition sensor75 may be used as a second means of patient identity confirmation inorder to confirm the identity of the patient that was previouslyverified by the image recognition sensor 73. In other words, the imagerecognition sensor 73 may comprises a first stage of patient identityconfirmation, and the voice recognition sensor 75 may comprise a secondstage of patient identity confirmation.

Similar to the image recognition sensor 73 described above, the voicerecognition sensor 75 of each illustrative local control system may beoperatively connected to the local computing device, which forms thecontrol-processing means 59 a of the local control system. The localcomputing device may be specially programmed with voice recognitionsoftware loaded thereon, and executed thereby for performing all of thefunctionality necessary to identify and verify a particular subject orpatient that is to be treated. Initially, the local computing device maybe specially programmed to capture and store a first reference speechwaveform of the subject or patient so that the first reference speechwaveform may be compared to a second speech waveform of the same patientor subject captured at a later time (i.e., just prior to the performanceof the surgical procedure). That is, the patient or subject may be askedto say a particular word or plurality words that are captured by thevoice recognition sensor 75 so that it can be used as the firstreference speech waveform. Then, prior to the performance of thesurgical procedure (e.g., a laser coagulation performed on the eye), thesecond speech waveform of the subject or patient is captured by thevoice sensor 75 (i.e., the microphone records the same word or pluralitywords repeated by the subject or patient) and the local computing devicecompares the second speech waveform of the patient or subject to thefirst reference speech waveform and determines if the second speechwaveform of the subject or patient matches or substantially matches thefirst reference speech waveform (i.e., by comparing the frequencycontent of the first and second speech waveforms). When the localcomputing device determines that the second speech waveform of thesubject or patient matches or substantially matches the first referencespeech waveform, the local computing device is specially programmed togenerate a matched speech confirmation notification that is sent to theremote computing device at the remote site in order to inform theattending physician that the proper patient has been identified andverified. The matched speech confirmation notification may also bedelivered to the technician at the local site via the local computingdevice. Then, after the other safety checks of the system 50′ have beenperformed, the surgical procedure is capable of being performed on thepatient. Conversely, when the local computing device determines that thesecond speech waveform of the subject or patient does not match orsubstantially match the first reference speech waveform, the localcomputing device is specially programmed to generate a non-matchingspeech notification that is sent to the remote computing device at theremote site in order to inform the attending physician that the patienthas not been properly identified and verified. The non-matching speechnotification may also be delivered to the technician at the local sitevia the local computing device. When the non-matching speechnotification is sent to the attending physician, the local computingdevice also disables the surgical equipment at the local site in orderto prevent the procedure from being performed on the incorrect patient(e.g., in a laser coagulation procedure, laser firing will beautomatically locked out by the local computing device).

Similar to that described above for the system 50 of FIG. 5, thelaser-imaging system 50′ of FIG. 15 also includes a central controlsystem 58 at a remote site having a command computer 59 b that isoperatively connected to a remote control module 64. Also disposed atthe remote site 58 during a laser procedure is a system operator (e.g.,retinal surgeon). In FIG. 15, it can be seen that the central controlsystem 58 at the remote site, which includes the command computer 59 b,is operatively connected to the plurality of local control systemsdisposed at local sites 51 a′, 51 b′, 51 c′ via a computer network thatuses the Internet 60.

In one or more further embodiments, laser-imaging system 50′ may furtherinclude a dynamic imaging system comprising an imaging device configuredto capture images of a body portion of a person over a predeterminedduration of time so that a displacement of the body portion of theperson is capable of being tracked during the predetermined duration oftime; and a data processing device (which may be the local computingdevice 59 a) operatively coupled to the imaging device, the dataprocessing device being specially programmed to determine thedisplacement of the body portion of the person over the predeterminedduration of time using the captured images, and to compare thedisplacement of the body portion of the person over the predeterminedduration of time to a reference displacement of the body portion of theperson acquired prior to the displacement so that dynamic changes in thebody portion of the person are capable of being assessed (e.g., by thelocal computing device 59 a and the software algorithm executed thereby)for the purpose of identifying the person or evaluating physiologicalchanges in the body portion. In particular, the local computing device59 a may execute a subtraction algorithm to demonstrate and project thespecific areas of the body or face and ignore the rest of the data toclearly present weighted pixelated data for the future comparison of thesame body or face and the trends of the area(s) of displacement. Inthese one or more further embodiments, at least two points ofdisplacement may be used for the comparison of the subsequentdisplacement of the body portion of the person to the referencedisplacement and its recorded displacement trend(s).

In one or more embodiments, the subtraction algorithm or analysisperformed by the local computing device 59 a may subtract a secondsubsequent image from a first image in order to isolate the portion ofthe second image that changed as compared to the first image. In otherwords, the subtraction algorithm or analysis performed by the localcomputing device 59 a removes or subtracts out the portion of the firstand second images that is unchanged between the two images so as toisolate only the changed image portion. In one or more embodiments, thedynamic imaging procedure is a two-step process. During the first step,a first image of the patient is taken (e.g., the face of the patient)without requiring the patient to perform any particular task (e.g.,smile or frown). During the second step, a second image of the patientis taken (e.g., the face of the patient) while requiring the patient toperform a particular task (e.g., smile or frown). Then, the first andsecond images are compared by the local computing device 59 a using thesubtraction algorithm or analysis. For example, if the patient is askedto smile or frown during the dynamic imaging procedure, only the mouthportion of the facial image of the patient is changed, thus thesubtraction algorithm or analysis performed by the local computingdevice 59 a eliminates the remainder of the facial image that isunchanged in order to enhance the processing speed of the image analysisprocess.

Because the second step of the two-step process dynamic imagingprocedure described above requires the patient to perform an action whenasked to do so by the clinician, the two-step dynamic imaging procedureresults in the patient giving implicit consent to the dynamic imagingprocedure (e.g., for identification purposes) because the person mustvoluntarily execute the task that is requested of him or her prior tothe second image being taken. As such, consent issues that arise withconventional patient identification methods are averted by the dynamicimaging system described herein.

In one or more embodiments, the local computing device 59 a is furtherprogrammed to determine a confidence level of the patient identificationprocedure performed by the dynamic imaging system herein. For example,the local computing device 59 a may determine with 98% certainty that asubsequent image is of the same patient as that depicted in an initialreference image. The local computing device 59 a may further beprogrammed to determine a percentage change between a first image and asecond image (e.g., a percentage change in the pixels of a first tumorimage and a second tumor image). The local computing device 59 a mayadditionally be programmed to reconstruct an incomplete image of apatient based upon a previous image of the same patient (e.g.,reconstruct an incomplete facial image).

In one or more further embodiments, the image recognition sensor 73 orimaging device of each local control system may be operatively connectedto the local computing device, which forms the control-processing means59 a of the local control system. The local computing device may bespecially programmed with image/pattern recognition software loadedthereon, and executed thereby for performing all of the functionalitynecessary to identify and verify a particular subject or patient, or toidentify and verify a body portion of the particular subject or patientthat is to be treated, the local computing device may be connected viathe internet to the remote location where the operator or a doctor canuse a helmet or gloves fitted with sensors or use the personal computer59 b with a mouse and software capable of creating an artificialenvironment where the patient or the subject is located (e.g. in aremote office as an augmented reality), so that the doctor or anobserver can obtain all the data from the patient's body, face, orretina, etc. prior to or during or after dynamically inducedphysiological changes in the patient or subject, so that operator or thedoctor can move the content of the three-dimensionally created imagefrom one to another direction or zoom in or out or using presentlyavailable technology, such as Google® Glass or a heads-up display with asmall screen in front of the eyes, etc. or as a wrap-around displayscreen as virtual reality to simulate the dynamic observation of thechanges, the trend and subtraction of the before and after images by thecomputer algorithm to look objectively at the final product for thefirst time or compare it in the future session to another dynamic imagesimilarly obtained from the patient or the subject to observe thedifferences or trends of the final subtracted data in the patient's orsubject's image in the same environment where the patient is located,and thus creating an augmented reality or an artificial environment asvirtual reality away from the patient or the subject and storing theinformation and data obtained after treatment of the patient in thecloud or another computer after encrypting the data for futurereference.

In these one or more further embodiments, the image recognition sensor73 or imaging device of each local control system may be operativelyconnected to the local computing device, which forms thecontrol-processing means 59 a of the local control system. The localcomputing device may be specially programmed with image/patternrecognition software loaded thereon, and executed thereby for performingall of the functionality necessary to identify and verify a particularsubject or patient, or to identify and verify a body portion of theparticular subject or patient that is to be treated. Initially, thelocal computing device may be specially programmed to capture and storereference dynamic information regarding a body portion of the subject orpatient so that the reference dynamic information may be compared todynamic information pertaining to the same body portion captured at alater time (i.e., just prior to the performance of the surgicalprocedure). Then, prior to the performance of the surgical procedure(e.g., a laser coagulation performed on the eye, or body surface, etc.),dynamic information pertaining to the same body portion of the subjector patient is captured by the image sensor 73 (i.e., the light fieldcamera) and the local computing device compares the subsequent dynamicinformation pertaining to the body portion to the reference dynamicinformation and determines if the subsequent dynamic information of thebody portion matches or substantially matches the reference dynamicinformation. In one or more embodiments, this information may relate tothe position or size of any part of the body, extremities, or organ(e.g., as seen on an X-ray film, MRI, CT-scan, or gait changes bywalking or particular habit of head position, hand, facial changesduring speech, or observation of changes due to emotional changes,etc.). When the local computing device determines that the subsequentdynamic information pertaining to the body portion of the subject orpatient matches or substantially matches the reference dynamicinformation, the local computing device is specially programmed togenerate a matched identity confirmation notification or the trends ofchanges, etc. in a two-dimensional and/or three-dimensional manner thatis sent to the remote computing device at the remote site in order toinform the attending physician or the receiving authority that theproper patient or body portion of the patient has been identified andverified or his or her response to stimuli. The matched identityconfirmation notification may also be delivered to the technician at thelocal site via the local computing device. Then, after the other safetychecks of the system 50′ have been performed, the surgical procedure iscapable of being performed on the patient. Conversely, when the localcomputing device determines that the subsequent dynamic informationregarding the body portion of the subject or patient does not match orsubstantially match the reference dynamic information, the localcomputing device is specially programmed to generate a non-matchingidentity notification that is sent to the remote computing device at theremote site in order to inform the attending physician that the patientor body portion of the patient has not been properly identified andverified. The non-matching identity notification may also be deliveredto the technician at the local site via the local computing device. Whenthe non-matching identity notification is sent to the attendingphysician, the local computing device also disables the surgicalequipment at the local site in order to prevent the procedure from beingperformed on the incorrect patient or the incorrect body portion of thepatient (e.g., in a laser coagulation procedure, laser firing will beautomatically locked out by the local computing device).

In these one or more further embodiments, turning again to FIG. 15, itcan be seen that each local control system may further include a voicerecognition sensor 75 for capturing the speech sound waves generated bythe subject or patient so that the speech of the subject or patient mayadditionally be used to identify and/or verify the identity of thesubject or patient, and his or her gender according to the frequenciesof the sound waves and using, similar to the imaging system, asubtraction analysis of the sound initially and in the subsequentsessions. In one or more embodiments, the voice recognition sensor 75may be used in conjunction with the image recognition sensor 73 orimaging device described above to further verify that a surgicalprocedure is being performed on the correct subject or patient. Thevoice recognition sensor may be simultaneously overlaid on the dynamicimage recognition changes for facial or body recognition. Particularly,in one or more embodiments, the sound waves generated from the dataacquired by the voice recognition sensor 75 may be superimposed on thedisplacement curve generated from the data acquired by the imagingdevice (e.g., the light field camera) so that both audial and visualattributes of the person may be taken into account for identificationpurposes, thereby making the identification of the person far moreaccurate. For example, as a person recites a series of vowels (i.e.,AEIOU), the displacement of the lips of the person is recorded by theimaging device (e.g., the light field camera), while the voicerecognition sensor 75 simultaneously records the voice of the person(e.g., the amplitude and/or frequency of the sound wave generated by theperson). In one or more embodiments, the dynamic imaging system is usedalone or simultaneously with voice overlay, thereby creating 2D or 3Dimages using multispectral light, which includes IR and mid IR, tomeasure time dependent changes for creation of a dynamic event made ofvoice and images or changes thereof. In one or more embodiments, asubtraction algorithm of the system is used to produce a clearsubtracted wave complex from a person examined a first and second time,and to project the subtracted wave complex on the subtracted value ofthe person's image so as to evaluate a match or change, and present thechanged values or their difference as compared to the sound wavesreceived the first time by the voice recognition sensor 75. In one ormore embodiments, the voice recognition sensor 75 may comprise amicrophone that captures the speech of the subject or patient over theentire speech frequency range of a human being (e.g., for a frequencyrange from 50 Hz to 5,000 Hz to encompass the typical frequency rangefor both males and females). As such, the syntax and sound frequenciesgenerated by the subject or patient are capable of being used by thelocal control system for verification and identification of the subjector patient prior to the surgical procedure being performed on him orher. In one or more embodiments, the voice recognition sensor 75 may beused as a second means of patient identity confirmation in order toconfirm the identity of the patient that was previously verified by theimage recognition sensor 73 or imaging device. In other words, the imagerecognition sensor 73 or imaging device may comprise a first stage ofpatient identity confirmation, and the voice recognition sensor 75 maycomprise a second stage of patient identity confirmation.

Similar to the image recognition sensor 73 or imaging device describedabove, the voice recognition sensor 75 of each illustrative localcontrol system may be operatively connected to the local computingdevice, which forms the control-processing means 59 a of the localcontrol system. The local computing device may be specially programmedwith voice recognition software loaded thereon, and executed thereby forperforming all of the functionality necessary to identify and verify aparticular subject or patient that is to be treated. Initially, thelocal computing device may be specially programmed to capture and storea first reference speech waveform of the subject or patient so that thefirst reference speech waveform may be compared to a second speechwaveform of the same patient or subject captured at a later time (i.e.,just prior to the performance of the surgical procedure). That is, thepatient or subject may be asked to say a particular word, a plurality ofwords, or a series of vowels (i.e., AEIOU) that are captured by thevoice recognition sensor 75 so that it can be used as the firstreference speech waveform. Then, prior to the performance of thesurgical procedure (e.g., a laser coagulation performed on the eye orthe body), the second speech waveform of the subject or patient iscaptured by the voice sensor 75 (i.e., the microphone records the sameword, plurality of words, or series of vowels repeated by the subject orpatient) and the local computing device compares the second speechwaveform of the patient or subject to the first reference speechwaveform and determines if the second speech waveform of the subject orpatient matches or substantially matches the first reference speechwaveform (i.e., by comparing the frequency content of the first andsecond speech sound waves). When the local computing device determinesthat the second speech waveform of the subject or patient matches orsubstantially matches the first reference speech waveform, the localcomputing device is specially programmed to generate a matched speechconfirmation notification that is sent to the remote computing device atthe remote site in order to inform the attending physician that theproper patient has been identified and verified. The matched speechconfirmation notification or its discrepancies may also be delivered tothe technician at the local site via the local computing device. Then,after the other safety checks of the system 50′ have been performed, thesurgical procedure is capable of being performed on the patient.Conversely, when the local computing device determines that the secondspeech waveform of the subject or patient does not match orsubstantially match the first reference speech waveform, the localcomputing device is specially programmed to generate a non-matchingspeech notification that is sent to the remote computing device at theremote site in order to inform the attending physician that the patienthas not been properly identified and verified. The non-matching speechnotification may also be delivered to the technician at the local sitevia the local computing device. When the non-matching speechnotification is sent to the attending physician, the local computingdevice also disables the surgical equipment at the local site in orderto prevent the procedure from being performed on the incorrect patient(e.g., in a laser coagulation procedure, laser firing will beautomatically locked out by the local computing device). In one or moreembodiments, the voice changes representing various emotional stages ofthe patients may be recorded for diagnosis or excessive stimuli, such asemotion or pain causing those changes, so that the power of a laser isaccordingly adjusted, etc.

In one embodiment, the local computing device is further programmed toask the patient a question so that the patient is able to respond to thequestion posed to him or her using natural language. That way, thesystem is not only able to analyze the physical surface attributes ofthe patient, but also analyze the sound of the patient's voice (i.e., byvoice recognition recording), and communicate simultaneously withaccessible existing data in a computer database to verify the identityof the patient.

In one embodiment, the time and the information obtained from thepatient is recorded and stored for the future recall. In one embodiment,for a patient whose history does not yet exist in the computer database,the information will be recorded so as to be recognized by the systemthe next time the patient comes for reexamination.

In one embodiment, the system can access other computers searching forthe existence of similar cases and photos of a surface lesion (e.g.,X-ray, CT-scan, MRI, PET-scan, etc. of a lesion). The system may also beused as a telesystem accessing existing data in the published literatureto assist the doctor with the diagnosis and further therapyrecommendation for the patient.

In one embodiment, the computer system functions as an informationalunit augmenting the knowledge of the doctor and assists in presentinghim or her with similar recorded cases to assist in a bettertelemedicine diagnosis and management.

In one embodiment, the system may augment recognition of the patient byusing additional information from a fingerprint, etc. For example,information regarding the tip of the patient's finger may be recorded(e.g., the blood circulation in the finger as well as images of theridges and minutiae, etc. of the fingerprint) in a two-dimensionaland/or three-dimensional manner so as to be used for identifying thepatient in conjunction with the dynamic facial recognition of the personin a two-dimensional and/or three-dimensional manner. As such, thesystem advantageously provides augmented dynamic identification of thepatient.

In one embodiment, the system works as an augmented intelligenceassistant so as to assist a person in to making a proper decision (e.g.,proper treatment of a cancer patient).

In one embodiment, the information obtained by the system is encryptedand transmitted so as to make it virtually impossible to be hacked by athird party.

In one embodiment, the system can differentiate between a human personand robot by its multispectral camera analysis recording the body'stemperature versus the relatively cold body of a robot and thereflective properties of its body surface, etc.

In one embodiment, the system can differentiate between a person havinga dark skin pigmentation and another person with lighter skinpigmentation by its multispectral camera analysis using the infrared(IR) spectrum of the camera to obtain images during the dynamic facialrecognition that can see the skin capillaries under the skin because ofselective penetration of IR wavelength of the light through the skinpigment and observe changes (e.g., collapse of capillaries at specificsites around the lips and in the eyebrows or forehead that fold and thecollapse the capillaries inside the folds) during the dynamic facialrecognition (e.g., with smiling or frowning), creating distinctlandmarks that can be recognized by the subtraction of images using thesoftware algorithm or observing additional landmarks, such as the teethduring smiling.

As mentioned above, in the illustrative embodiment, the imagerecognition sensor 73 or imaging device at the remote laser deliverysite may be in the form of a digital light field photography (DLFP)camera with microlenses that capture the information about the directionof the incoming light rays and a photosensor array that is disposedbehind the microlenses. A specially programmed data processing device(e.g., the local computing device 59 a) is used to process theinformation obtained from the light field camera by converting it intotwo or three dimensional images, or the data is transmitted to a virtualreality system to be observed and recorded by the examiner or a doctor.

In one or more embodiments, the light field digital camera or digitallight field photography (DIFLFP) camera comprises one or more fixedoptical element(s) as the objective lens providing a fixed field of viewfor the camera. A series of microlenses are located at the focal pointof the objective lens in a flat plane perpendicular to the axial rays ofthe objective lens. These microlenses separate the incoming rays oflight entering the camera into individual small bundles. The individualsmall bundles of light are refracted on a series of light sensitivesensors that measure in hundreds of megapixels, which are located behindthe plane of the microlenses, thereby converting the light energy intoelectrical signals. The electronically generated signals conveyinformation regarding the direction of each light ray, view, and theintensity of each light ray to a processor or a computer. Each microlenshas some overlapping view and perspective from the next one which can beretraced by an algorithm. Appropriate software and algorithmsreconstruct computer-generated 2-3 D images from the objects not only infocus, but also those located in front or in the back of the object from0.1 mm from the lens surface to infinity, which is being photographed byretracing the rays via the software and algorithm that modifies, ormagnifies the image, as desired, while eliminating electronically theimage aberrations, reflections, etc. As such, by using a digital camera,one can manipulate the image data by using an appropriate algorithm soas to create new in focus image(s) mathematically.

In one or more embodiments, the light sensitive sensors behind thelenslets of the camera record the incoming light and forward it aselectrical signals to the camera's processor and act as an on/off switchfor the camera's processor measuring the intensity of the light throughits neuronal network and its algorithm to record changes in lightintensity, while recording any motion or dynamic displacement of anobject or part of an object in front of the camera in a nanosecond to amicrosecond of time. The processor of the camera with its neuronalnetwork algorithm processes the images as the retina and brain in ahuman being functions by finding the pattern in the data and its dynamicchanges of the image and its trend over a very short period of time(e.g., nanosecond). The information is stored in the memory system ofthe camera's processor, as known in the art, as memory resistor(memristor) relating to electric charge and magnetic flux linkage, whichcan be retrieved immediately or later, and further analyzed by the knownmathematical algorithms of the camera and can be used for many differentapplications, in addition to applications for 2-3 D dynamic imagerecognition as near or remote subjects recognition, incorporated in theremote laser delivery system described above.

In one or more embodiments, the light field camera may have either atunable lens or a fluidic lens. If a tunable lens is utilized, thetunable lens may be in the form of a shape-changing polymer lens (e.g.,an Optotune® lens), a liquid crystal lens, an electrically tunable lens(e.g., using electrowetting, such as a Varioptic® lens). Alternatively,a fluidic lens may be used in the light field camera.

In one or more embodiments, the light field camera may have a simpleflexible transparent membrane that acts like a lens where its surfaceconvexity or concavity can be controlled by a mechanically orelectronically by compressing or decompressing a deformable lens by anon-deformable object, such as a magnetic plate or a piezoelectricsystem.

In one or more embodiments, the fluidic lens of the light field camerais a soft flexible lens which can be deformed electronically by applyingpressure on the lens to increase or decrease the refractive power of thelens.

In one or more embodiments, the fluidic or tunable lens of the lightfield camera is combined with a solid lens, thereby creating a hybridlens providing a minus or plus addition to the tunable lens.

In one or more embodiments, the fluidic light field camera shutter actslike a lid that covers the lens surface and is moved only up and down,or in horizontal direction to open or close the shutter, while in anopen phase, the lens is exposed or is bulged outward permitting a widerfield of view than the standard shutters provide.

In one or more embodiments, the tunable light field camera can bemonocular or binocular camera and the images can be used in augmentedreality (AR) or virtual reality (VR). The combination of two or moretunable field cameras can create 360 degrees in a 3-D format innon-dynamic and dynamic identity recognition.

In one or more embodiments, tunable light field camera with its softwareand algorithm can be used for dynamic recognition of an object, animateor inanimate, animal or human, where the dynamic changes can be observedin a 3-4 dimensional manner depending on which direction one likes toobserve the subject or a person which now can be presented with AR/VRglasses or subtracted to provide information about the changes that hasoccurred during the dynamic observation.

In one or more embodiments, the tunable light field camera can be usedfor producing videos in augmented reality (AR) or virtual reality (VR)and can be used with appropriate AR/VR glasses while simplifying theobservation of the image since the camera provides a focal point for anyarea which is being observed, thereby eliminating potentially the needfor convergence while seeing the object monocularly or binocularly.

In one or more embodiments, the combination of a tunable light fieldcamera and augmented reality (AR) or virtual reality (VR) can be used inthe medical field to virtually recognize the growth of a lesion (e.g., atumor) or the area of damage caused by a disease or trauma, etc. In thisapplication, one will be able to see the structure in three and fourdimensions during an invasive procedure, such as surgery, or a minimallyinvasive procedure through an endoscope, or a completely non-invasiveprocedure such as thermotherapy, using laser, focused ultrasound, or analternating magnetic field along with antibody coated magneticnanoparticles administered intravenously to seek the tumor cells andmake them subsequently visible by CT-scan, ultrasound, or MRItomography, or using the tunable light field camera and VR during theradiation therapy, laser therapy, or focused ultrasound, etc.

In one or more embodiments, the tunable light field camera and VR can beused by architects to see a building in 3D-4D format before it is builtand its structural stability is examined with augmented reality (AR)glasses or virtual reality (VR) glasses.

In one or more embodiments, using the tunable light field camera alongwith augmented reality (AR) or virtual reality (VR), architects are ableto see a building in 3-4D format before it is built and its structuralstability is examined with AR/VR glasses (e.g., in AR/VR during anearthquake, hurricane, tornado, etc.) produced in vitro to evaluatestability of a high-rise building, bridge, airplane, or any otherstructure, etc. or any area that requires 3-D, 4-D, or 5-D observationand planning (e.g., airline pilots to see their surrounding area toavoid an aerial crash by another plane or collision with birds, etc.).

In one or more embodiments, the light field camera is used as a part ofa dynamic facial recognition system for patient identification andverification, as described above. Advantageously, the tunable lens, whencombined with the other components of the light field camera, offersprecise focusing using the microlenses, nanosensors, and computer toanalyze numerous focal points that can be reconstructed mathematicallyusing specific algorithms implemented by the data processing device(i.e., the computer of the dynamic imaging system). Also, the dynamicimaging system may use the computer to verify or identify variouschanges that happen during the change in physiological function of aperson's facial expression (e.g., smiling or frowning) as acomputer-generated digital 2D or 3D image or video frame records thedynamic changes of a structure, such as a face, mouth, eyes etc., andthe computer analyzes and compares the biometrics as a dynamicphysiological fingerprint with existing data of the same image. Thecomputer algorithm analyzes the changes in the relative position of apatient's face, matching points and directions, and compressing the dataobtained during the process using dynamic recognition algorithms. Oneexemplary technique employed is the statistical comparison of the firstobtained values with the second values in order to examine the variancesusing a number of means including multi-spectral light that looks at thevarious physiological changes of the face. Mathematical patterns of thedigital images and statistical algorithms are capable of demonstratingthat the images obtained initially and subsequently belong to the sameperson, etc. For example, the computer may compare the images usingknown techniques, such as elastic bunch graph matching, face recognitionusing a Fisherface algorithm, face recognition using dynamic linkmatching, etc. Also, these techniques may be employed by the dynamicimaging system for exploratory data analysis to predict the changes ofthe image (e.g., aging of a face or tumor growth, etc.). For example, adynamic analysis of the growth of a tumor may be performed using thedynamic imaging system described herein (e.g., by analyzing the increasein the surface area or the volume of the tumor). That is, using thesystem described herein, the volumetric changes of a tumor or lesion iscapable of being measured over a time period by software subtractionalgorithms of the system as explained below, and then transmitted to thetreating physician. In addition, the dynamic changes in a portion of apatient's body may be compared with existing data for diagnosis ordifferential diagnosis so as to track and analyze trends in diseaseprogression or disease improvement against baseline data for managementof diseases. The dynamic imaging system described herein may beconfigured to track changes in the disease process (e.g., diabeticretinopathy, another retinal disease, brain, spinal cord vertebrae,prostate, uterus, ovarian, intestine, stomach, extremities, lung, heart,skin disease, eczema, breast cancer, a tumor in the body, etc.) over aperiod of time so that the disease process is capable of being monitoredby a physician. Also, follow-up images may be acquired using X-ray,CT-scan, positron, MRI, ultrasound, or photoacoustic imaging, etc.

In one or more embodiments, the body portion of the person being trackedby the imaging device comprises a tumor, lesion, retina, lung, or heart,and the dynamic imaging system is configured to dynamically analyze thegrowth of the tumor or lesion over a period of time by tracking thedisplacement of the tumor or lesion in a two-dimensional orthree-dimensional manner over the period of time.

In one or more embodiments, a grid is projected over the area ofinterest in the dynamic facial recognition system to clearly define theposition of each pixelated area of the face to compare it with the imageor displacement that has occurred in the process, and to superimposeimages on each other by the computer executing a dynamic subtractionsoftware algorithm to demonstrate the degree of the change and its trendduring the displacement or change, thereby presenting it as a subtractedimage when using a multispectral camera capable of analyzing the widespectrum of the wavelength (images) including the infrared or nearinfrared wavelengths and analyze them by the software of the camerarapidly presenting it as 2-D or 3D images (refer to FIGS. 18a and 18b ).

FIGS. 18a and 18b depict a grid 506 that has been projected over thefacial area of a person 500 in order to track dynamic changes in thelips 508 of the person 500. In FIG. 18a , the positions of two exemplarypoints 502, 504 are being used to track the dynamic changes in the lips508 of the person 500. In FIG. 18b , the positions of two exemplarypoints 512, 514 are being used to track dynamic changes in the lips 508of the person 500 while the person 500 is speaking the letter “O”. Asshown in FIG. 18b , as the letter “O” is spoken by the person 500,grooves 510 are formed in the skin of the facial area surrounding thelips 508.

When the system is equipped with a multispectral camera, themultispectral camera may be used to obtain photos either in the visiblespectrum or infrared to low infrared light spectrum working as athermographic camera, seeing deep inside the skin to recognize thestatus of the circulation under the skin. The infrared patternrecognition capabilities of the system record psychological functionalchanges occurring under the skin (such as an increase or decrease in thecirculation due to sympathetic activation-deactivation) together withdynamic changes, which are not achievable with a camera having onlyvisible light capabilities.

In one or more embodiments, the visible spectrum provides informationfrom the surface structure during the dynamic facial changes caused bydeliberate activation of the facial muscles producing skin groovesaround the mouth and the eye demonstrating the physical aspects ofchanges in a person being photographed in a two-dimensional and/orthree-dimensional manner.

The computer software of the system analyzes both of the aforedescribedfacial changes and presents them as independent values that can besuperimposed mathematically by the computer's software creatingsubtraction data indicating the changes that have occurred serving asthe initial face interrogation data. This subtraction algorithm may beused subsequently for recognition of a face, body, a tumor located onthe surface of the skin or inside the body imaged to recognize theextent of changed values in two or three dimensional format. The dynamicimaging system described herein may also be used also along withstandard imaging systems such as X-ray, CT-scan, MRI, positron, OCTimaging, photoacoustic imaging, thermoacoustic imaging, or optoacousticimaging to record changes occurring in the images and its temperatureafter treatment over a time period. Because the images are pixilatedbits of information that can be recorded live (e.g., for comparing animage before surgery and after the surgery), the images can besubsequently subjected to analysis with the subtraction algorithm of thecomputer software to decide, for example, if a tumor is completelyremoved or not.

In one embodiment, the algorithm and the software of dynamic imagerecognition system can be used to analyze the changes that can occurusing artificial intelligence (AI) with a standard image technique suchas X-ray, CT-scan, MRI, ultrasound, photoacoustic tomography,microscopy, optical coherence tomography (OCT), or confocal microscopy,etc. to diagnose early minute changes that occur in the body or tissueas a result of a disease process, such as macular degeneration,diabetes, heart or vascular disease, urogenital diseases, liver, kidney,or gastrointestinal diseases, extremities and joint diseases, skindiseases, Alzheimer's disease, Parkinson's disease, or othercerebrovascular diseases.

As another example, when the subject frowns, skin capillaries becomecollapsed and folded, thereby reducing the blood flow through thecollapsed and folded capillaries and the heat that is detected by themultispectral camera. The subtraction algorithm provides the exactchanges in facial structure during the frowning of the subject.

In one or more other embodiments, the subtraction algorithm executed bythe computer presents only the subtracted image and its mathematicalweight or value, and compares it with the previously obtained subtractedimage of the same structure or face to verify the identity the personand compare the initial values and the extent of the changes between thefirst and second captured images, and the trend of the changes that haveoccurred after displacement of the surface or a structure of interest,such as a tumor dimension over time and its growth trend.

In one or more embodiments, the before or after image subtraction may beperformed during the imaging of a tumor (e.g., a brain tumor) usingcontrast agents or antibody coated nanoparticles to demonstrate thedegree of involvement of the brain and the effect of surgery or therapyon the 3-D dimension of the tumor indicating whether the tumor is ableto be removed or treated completely or not. In one or more embodiments,the same computer algorithm is used in machine vision, robotic vision,or drone vision to examine the before and/or after effect of an actionover time and predict the trend.

In one embodiment, in dynamic facial recognition, a conventionalexisting mathematical algorithm is not used to compare two static imagesand conclude the possibility or probability of them being the same, butrather the computer of the present system is specially programmed tocompare two sets of dynamic images, which are composed of one static andone dynamic, that add significantly more information by dynamic changesin a two-dimensional and/or three-dimensional manner that have occurredas a result of displacements of various points (e.g., in the face of theperson) and the trend of the changes obtained by the computersubtraction of the dynamic changes, where to this, two significantvalues that augment the weight of the computer-generated image bysuperimposition of the obtained data or pre and post dynamic images forsubtraction analysis by adding the voice or sound waves recordedsimultaneously or superimposed over the dynamic facial values, andfinally confirming the data with dynamic fingerprinting that has twoadditional components of finger print analysis, and multispectralphotography before and after pressing the finger over the transparentglass, and collapsing the capillaries of the hand or the finger thatprovides practically a complementary algorithm for nearly infallibleidentity recognition data of a person within a timeframe of 2½ secondsor less. For example, FIG. 26 depicts a graphical representation 636 ofthe correlation between the mouth configuration and the sound wavegenerated by a person while speaking the words “I love you”. The firstsection 638 of the sound wave corresponding to the speaking of the word“I” produces an open mouth configuration 644 with wrinkles on the lipsof the mouth. The second section 640 of the sound wave corresponding tothe speaking of the word “love” produces a semi-closed configuration 646of the mouth where the lips touch the teeth. The third section 642 ofthe sound wave corresponding to the speaking of the word “you” producesa semi-closed, generally round configuration 648 of the mouth.

In one embodiment, the dynamic facial recognition technology describedherein uses a software program that not only subtracts the data obtainedin the first session from the data obtained in the second session, butsimilarly through its subtraction algorithm, compares the sound wave ofa person at each examination setting to show the difference or thetrend, along with final confirmation of the data with dynamicfingerprinting technology of pressing a finger or palm of the hand overa transparent glass which is photographed before and after pressing thefinger/palm with multispectral light waves to demonstrate the finger'sor palm's circulation pattern, or its collapse after pressing it overthe transparent glass, in addition to the ridges of the skin, folds,minutiae, etc. in a two-dimensional and/or three-dimensional manner fordynamic identity recognition.

The technology described herein demonstrates a way to subtract theinformation of a dynamic change mathematically from a dynamicrecognition data of not only the face, but also extremities or a movingperson or variation of the facial folds measured by a multispecteral orhyperspectral camera or color changes of the collapsed capillariesobserved after pressing the finger or palm, or changes of the facialcapillaries during an interrogation, or observation of a joyful or sadmovie, etc., which all add value to the correct identification of aperson.

In one embodiment, the subtraction algorithm may also be used to predicta trend using one of the existing mathematical algorithms describedbelow, and recognize changes or the trend for evaluation of dynamicimaging, and another dynamic imaging related to the first set, such asadding a value or one dynamic facial recognition with another onecreated by pressing of the face to bleach the facial capillaries orinducing an artificially induced element that changes the facial folds(e.g., created by frowning to that of the voice and dynamic changes ofthe mouth by speaking a vowel or smiling, etc.) in a two-dimensionaland/or three-dimensional manner.

In one embodiment, the accuracy of the dynamic facial recognition systemmay be augmented with the use of another existing mathematicalalgorithm, such as an artificial neuronal network, interval finiteelement analysis, fuzzy logic variations, machine learning, rough set,Sorties Paradox, vector logic, etc.

In one embodiment, the camera of the dynamic facial recognition systemis equipped with software for machine learning, with a neuronal networkand medical imaging or perception, natural language, speech recognition,big data decision-making of the changes that occurs in a dynamic formator anthropology software, or artificial intelligence capable ofreconstructing a complete image from a part of the image or informationthat is obtained from a person.

In one embodiment, the person being imaged or photographed is an activeparticipant in the process that implies the person in giving his consentby participating in the process and his photograph is not taken withouthis permission randomly by a camera located in a location.

In one embodiment, standard facial recognition may be used with atelemedicine system despite its shortcoming of the person's recognitionand the fact it does not provide or imply a person's consent, by using arecording to record the person's consent for telemedicine communicationby asking the person to indicate his or her consent by responding to thequestion if she or he would give permission to discuss his or her healthor other issues by indicating “yes” or “no”, or any other meansvoluntarily while being recorded by a camera or voice recognitionsystem, etc.

In one embodiment, the person writes “yes” or “no” while the computerrecognizes the handwriting with the handwriting recognition, or theperson presses over an area of the monitor of a computer where thecomputer has selected indicating “yes” or “no” where the computer canrecognize the person from the minutiae of the tip of her or his fingeralong with dynamic recognition of the capillaries, or simultaneouslyobserving the dynamic changes of the person's face, etc. (i.e.,recording two or more independent tasks for recognition).

In one embodiment, the dynamic recognition is used in telemedicine, andone of its function is to secure a person's consent in telemedicine thatinvolves asking a person for his consent to take a photo or dynamicimaging from him or any of his or her body part and obtaining his or herconsent verbally or by head or hand or eye finger motion, etc., oradding his fingerprint to a surface of the computer or a smartphone thatcan record it in a 2D or 3D form or in a dynamic format with imaging thechanges in the fingertip or the organs or capillaries or using a sensorto simultaneously measure the pulse rate, etc. of the person, while thevoice recognition software records the sound waves of a patient in hisnative language or another language for future verification, and thesound waves are recorded, stored and encrypted.

In one embodiment, the dynamic image recognition is used in combinationwith computer artificial intelligence (AI) software and a camera thatmimics human recognition by learning and solving problems includingidentity of a person or a structure or a lesion, its dynamic changes anddata searching, calculating, sensing the environment in depth orsensing, the social intelligence using linguistic, psychology,mathematic and neuroscience to create new images and recognize theemotional changes of a person or animal.

In one embodiment, the dynamic image recognition system is capable oflearning algorithms that can recall, learn, and infer changes inrecognizing a person, lesion, etc. in any image presented with standardphotography or hyperspectral imaging, X-ray, MRI, or CT-scan, microscopyor confocal microscopy or OCT, etc. using hierarchal temporal memorysoftware.

In one embodiment, the dynamic image recognition system uses themathematical means of fuzzy logic or vagueness in a complex situationfor recognition of difficult images to differentiate certain persons(e.g., twins, etc.), along with dynamic identity recognition and machinelearning, speech recognition, to differentiate the effect of theenvironment on the person or animal or any other living structure.

In one embodiment, the dynamic identity recognition system uses theartificial intelligence (AI) with various existing softwareinstructions, algorithms, such as dimensional reduction, structuralprediction, anomaly detection, artificial neuronal network, parallelconstraint satisfaction, or parallel distributed processing, etc.

In one embodiment, the dynamic identity recognition is used with roboticmachine vision for macroscopic or microscopic differentiation of atleast two similar or dissimilar objects, the position, and 3-D format inorder to perform a task precisely as instructed by the software.

In one embodiment, the tunable field camera can have a tunable lens orthe one fluidic lens.

In one embodiment, the tunable lenses are tunable lens withshape-changing polymer lenses (i.e., Optotune®), liquid crystal lenses,electrically tunable lenses, electrowetting lenses (Varioptic®), or thefluidic lenses described by Peyman in U.S. Pat. Nos. 7,993,399 and8,409,278, etc.

In one embodiment, the tunable light field camera can be an integralpart of the laser coagulation system or be provided separately as acamera.

In one embodiment, a tunable light field camera or other field camerascan be used with the tele-laser system or separately, or they can alsobe used as a part of the facial recognition system as a dynamic facialrecognition system. Tunable light field cameras offer a precise focusingconcept using the microlenses, nanosensors, and a computer to analyzevarious focal points that can be created mathematically using specificalgorithms.

In one embodiment, the tunable light field camera uses the computer toverify or identify various changes that can happen during the change inphysiological function of a person, such as smiling or frowning, as adigital 2 D or 3-D image or video frame recording the changes andanalyzing and comparing the biometrics as a dynamic physiologicalfingerprints with the existing date of the same image (e.g., a face,etc.) using mathematical dynamic pattern recognition. The computeralgorithm analyzes the changes in the relative position of a patient'sface by matching points and directions and compressing the data obtainedduring the process using dynamic recognition algorithms.

In one embodiment, the dynamic identity recognition uses a geometricalgorithm to distinguish the features of a person or a lesion, etc., andphotometric analysis provides precise statistical data from the imagesto compare the data template e-variances.

In one embodiment, a retinal scan provides not only the identity of aperson evaluating a unique pattern of a person, but can be used toanalyze the changes in the thickness of the retinal structures, alongwith OCT and analyze the changes of the retinal capillaries and retinalpigment epithelium, etc. using the known principal component analysis orlinear discriminant analysis (LDA), generalization of Fisher's lineardiscriminant, pattern recognition, and machine learning to provideprognostic information (e.g., in age related macular degeneration ofdiabetic retinopathy, glaucoma, and other diseases causing retinaldegeneration or abnormal blood vessels).

In one embodiment, in dynamic identity recognition (DIR), one uses 3-Dimages that are created for skin texture analysis by projectingstructured light on the skin, mucosa or retina, etc. (e.g., for asecurity system at an ATM).

In one embodiment, the computer modifies the 3-D image of a person takeninstantaneously or an existing image and modifies it with its artificialintelligence (AI) system for a search of similar images in its file orthe computer can modify the person or animal image by adding orsubtracting certain variable components to the person, such as addingdifferent types of hair including color, etc., or remove the hair,changing the facial hair, adding or subtracting earrings, changing theskin color or the color of the iris, adding or subtracting glasses, etc.By confirming the stable points in an image (e.g. distances between theeye, nose, mouth, ear, etc.—see e.g., stable features 614 in FIG. 22) toconfirm the identity of the person being examined, these tasks can beperformed rapidly by a processor.

In one embodiment, the dynamic identity recognition system can add orsubtract, using software and artificial intelligence (AI), a 3-Ddistinct feature after it has checked a person with the dynamicrecognition system (i.e., before and after dynamic images and computergenerated modifications). This computer software has the ability toartificially create an instant match with the person that is beingexamined by creating hundreds or thousands of similar images (e.g., byadding hat or glasses or changing the hair color of the subject and lookfor comparison in its previously stored memory). As shown in FIGS.20-25, the specially programmed computer of the dynamic identityrecognition system is able to eliminate the beard or hair of a personwhile evaluating other stable features or dynamically changed featuresof the same person, including the voice of the person, etc. For example,in FIG. 20, an initial image of a person 600 depicts the person withhair 602, a moustache 604, and a beard 606. In FIG. 20, the mouth 608 ofthe person is in a closed configuration. Turning to the image 610depicted in FIG. 21, it can be seen that a dynamic change in theappearance of the person has taken place. More specifically, a dynamicchange in the lower section of the person's face has occurred (i.e., themouth configuration 608′ is different from that of FIG. 20, resulting inenhanced folds on the face and visible teeth), while the midsection ofthe person's face has remained generally stable without any significantchanges.

In one embodiment, the software of the dynamic identity recognitionsystem can change the color of the skin, hair, or beard of the subjectusing all possible colors of the light spectrum, or adding hats or hair,toupee hair pieces or mustaches, or earring(s), etc. and/or convertingthe face of the person or patient to an emotional face showing, beingsad, being happy, or being in pain, etc. using the appropriate existingfolds and skin capillary color, fold compression enhancing or decreasingthe capillary circulation creating specific color intensity specific tothe emotional and physiological state of the subject's feelings, such aswell-being, using an infrared (IR) and visible light system in 3-D alongwith AI, AR, and VR software, using the voice recognition software forthe subject's voice and variation of sound frequency and amplitude torecognize the physiological state of the subject. For example, in FIG.22, a computer-modified image 612 of the person is depicted where thehair 602 of the person has been removed, thus resulting in a bald head616. Also, the facial hair of the person has been removed, therebyresulting in a hairless chin 618. The computer-modified image 612 inFIG. 22 enables the dynamic identity recognition system to bespoof-proof by enabling the system to still accurately identify theperson even if the person shaves his head, and removes his facial hair.In the case of such changes in the appearance of the person, the dynamicidentity recognition system is still able to accurately identify theperson by focusing on the stable features 614 of the person. Turning tothe image 620 depicted in FIG. 23, it can be seen that a dynamic changein the appearance of the person has taken place. More specifically, adynamic change in the lower section of the person's face has occurred(i.e., the mouth configuration 608″ is different from that of FIG. 22,resulting in enhanced folds on the face and visible teeth), while theupper section and midsection of the person's face has remained generallystable without any significant changes. In FIG. 24, anothercomputer-modified image 622 of the person is depicted where glasses 624have been provided on the person, the iris color 626 of the eyes of theperson has been changed, and an earring 628 has been added to one of theears of the person. In order to make the dynamic identity recognitionsystem spoof-proof, other computer-generated anatomical and cosmeticvariations can also be made to the person, such as changing the skincolor of the person, adding a hat on the head of the person, addinglipstick to the person, changing hair color, changing facial hair color.These computer-generated changes to the image of the person enhance thelikelihood of an identity match by the system even when the personalters his or her appearance (e.g., by shaving his head, shaving hisfacial hair, etc.).

In one embodiment, the dynamic identity recognition system can alsosimultaneously recognize via the software executed thereby the emotionalchanges that are associated with the existing folds and capillaries ofthe skin of a person's face that are enhanced by the emotional changesand can be subtracted from the existing prior images of the person orthe patient, etc. In FIG. 25, another computer-modified image 630 of theperson is depicted where an artificial change in the appearance of theperson has occurred while the person is speaking one or more words(i.e., represented by the sound wave 634 in FIG. 25). As shown in FIG.25, the computer-modified image 630 simulates a dynamic change in thelower section of the person's face while the person is speaking the oneor more words (i.e., the mouth configuration 608′″ is different fromthat of FIG. 24, resulting in enhanced folds on the face and visibleteeth). Also, in FIG. 25, simulated hair 632 has been added to theperson to increase the probability of an identity match if the persongrows hair after being bald.

In one embodiment, the dynamic identity recognition system can diagnosethe emotional changes by the use of its camera's imaging in the visiblespectrum or in the infrared (IR) spectrum.

In one embodiment, the computer of the dynamic identity recognitionsystem can record and subtract the emotional changes instantaneouslyinduced by conditions, such as anger or smiling, sickness, pain, orasking a question or remembering a past experience or trauma, etc.

In one embodiment, the computer of the dynamic identity recognitionsystem presents the before and after images and the subtracted one(e.g., from the original person's file to diagnose the physiological oremotional state changes of the subject).

In one embodiment, the dynamic identity recognition system eliminates orcreates spoofs for potentially creating a semi-virtual representation ofthe person with arbitrarily changing certain characteristics (e.g., thehair for matching the images of the subject for comparison purposes, orrecognizing the dynamic emotional changes of the subject both useful inthe medical field for diagnosis of an existing psychological conditionor use in a security system recognizing the person's response to, forexample, a question or a joke recognizing the physical and emotionalchanges in the images in 2-3-D format—see e.g., FIG. 25).

In one embodiment, all photos can be observed with specific glasses orgoggles as AR or VR. Since, the images are taken in 3-D, they can beobserved in 3-D and from any direction and volumetric analysis of theobject (e.g., head, nose, ear, chin, or a tumor, etc. can be compared oranalyzed).

In one embodiment, the computer of the dynamic identity recognitionsystem can access any library of images for obtaining existing priorimages or information, or similar images including clinical ones such asX-ray, MRI, CT-Scans, or photographs, etc. for analysis etc.

In one embodiment, a computer with dynamic identity recognitioncapabilities and artificial intelligence (AI) can generate from a personor animal a present image and a new computer-generated older or youngerperson's secondary image as long as certain elements or similarities arediscovered in the initial image to identify in the 2 to 3 dimensionalimage of the present image using AI and artificial neural networks (ANN)to create a new secondary image to build on the bases of unchanged partsof the present image (e.g., stable features 614 in the image 612 of FIG.22), and alternative images with the same basic characteristics foridentification using a software similar to those utilized inanthropology, to discover a whole structure from only a part of theimage or generating identifying characteristics from the examples(present image) that it processes.

In one embodiment, the software and algorithm of dynamic identityrecognition (DIR) and artificial intelligence (AI) can be used in therecognition of a person or an animal, or a person lost because of amental disease, such as Alzheimer's disease, or in a forensic situation.

In one embodiment, the software algorithms of dynamic identityrecognition (DIR), artificial intelligence (AI), and artificial neuralnetworks (ANN) with distributed computing and deep learning or thebi-directional and multi-dimensional long short-term memory (LSTM) orconvolutional neural networks (CNN), etc. can be used in systemidentification or control in trajectory prediction in microscopic ormacroscopic fashion, general game playing, 2-3 D pattern recognition ofinanimate objects or live humans, animals and other living things,signal classification, voice recognition, winking, gesture, or otherfacial expressions, etc., word or sounds expression of good or bad,handwriting recognition, medical, recognition of a disease process suchas cancer, before and after surgical procedure for tumor or inorthopedic surgery etc., visualization, microscopy, recognition, such ascells, tissue samples, bacteria, fungi, viruses, in swab, biopsy or intissue culture with simultaneous quantification.

In one embodiment, the dynamic identity recognition system utilizescomponents of artificial intelligence (AI) including training,validation with image and data, combined with a technical network, suchas a convolutional neuronal network (CNN). The training includesrecognition of obvious characteristics (e.g., such as face, eyes, nose,mouth, etc.) requiring focused images in 2D to 3D format or dynamicallychanged images that provide additional data along with voice recognitionwith deep learning algorithms such as CNN, VGGNet, AlexNet, InceptionV4, DenseNe, and ResNet.

In another embodiment, in a clinical setting, the dynamic identityrecognition system utilizes components of artificial intelligence (AI)in evaluation of a disease process in addition to what is describedabove for identity recognition of an individual. Artificial intelligence(AI) requires a standard or normal reference and means of comparison byhuman experts along with data collected from a large number of subjects.The machine learning or deep learning in artificial intelligence (AI)includes data obtained from clinical presentation, images and additionalmultimodal data, such as laboratory information used with deep learningalgorithms, such as neuronal computational layers available as CNN,VGGNet, AlexNet, Inception V4, DenseNe, and ResNet.

In another embodiment, the multispectral or hyperspectral camera is usedto obtain both spectral and spatial data from the images before andafter the dynamic changes have occurred on an external area of thesubject. In one embodiment, the characteristics and changes of a movingobject is recorded and analyzed providing simultaneous imaging,processing, and evaluation by the software of the processor of thecamera, thus sensing the simultaneous changes that are happening and thetrend of the changes (e.g., on the surface or below the surface of theskin or a subject being photographed). In one embodiment, the dynamicimages obtained from the surface can be combined with the informationobtained by the ultrasonic unit of the laser system to provideadditional information from a deeply located, internal body structure,such as bone, joints, or a moving object, etc. In one embodiment of theremote laser system, the electronically obtained images are combinedwith CMOS image sensors (e.g., analyzing a subject's fingerprint cangive information on the blood flow of the fingertip before or afterapplying pressure with the finger that collapse the fingers skincapillaries and the changes may be analyzed in real-time).

For example, a dynamic fingerprinting and/or dynamic hand recognitionsystem will be described with reference to FIGS. 19a-19c . Initially, asshown in FIG. 19a , a finger 516 containing numerous ridges and folds isplaced on a surface of transparent glass 518 so that the finger is ableto be imaged (i.e., photographed or videoed) with an infrared spectrumand/or visible spectrum of a field camera, multispectral camera, orhyperspectral camera 520 of the remote laser system. Rather than simplyimaging the fingertip in FIG. 19a , the entire hand of the person alsomay be imaged using the camera 520. The system of FIG. 19a may alsorecord the color of the fingertip or hand of the person prior to itsplacement on the surface of transparent glass 518 together with thefolds and ridges of the fingertip and hand.

Then, turning to FIG. 19b , a finger 522 is illustrated prior totouching the surface of transparent glass 518 and being imaged by thefield camera, multispectral camera, or hyperspectral camera 520 of theremote laser system. FIG. 19b depicts the fingertip or ball of thefinger 522 with its circulations, ridges, and minutiae, which are ableto be imaged using the camera 520 for highly reliable identification ofthe person. The infrared spectrum of the camera 520 is able to recordthe warm circulation of blood through the fingertip or ball of thefinger 522. FIG. 19c shows the ridges of the fingertip of the finger522, but centrally, the capillaries of the finger 522 are collapsed atthe area where the fingertip or finger ball is touching the surface oftransparent glass 518, which indicates that a live person is beingrecorded. In FIG. 19c , where the capillaries collapse at the ball ofthe finger 522, the finger 522 becomes devoid of circulation (thusturning red in color) at the site of compressed area compared with theprevious image; these changes are present in infrared photography.Advantageously, the system of FIGS. 19a-19c preserves the folds in thefinger 522 or, if the whole hand is placed on the glass 518, the foldsin the whole hand. In the illustrative embodiment, all of thisinformation is recorded before and after placement of the finger 522 orhand on the glass 518, and the changes are subtracted to obtain theverification of the person's identity, and the physical andphysiological changes that have occurred are analyzed to recognize andverify the person's identity.

In one embodiment, the person's finger, hand, or other extremities arevideoed to create more complete information regarding the identity ofthe person or the area of the body recorded for future dynamiccomparison, etc. In addition, a voice instruction may be included in thesystem of FIGS. 19a-19c in order to ask the person to press harder, orloosen up his or her finger, so that the degree of the capillarybleaching is confirmed, etc.

Also, in one or more embodiments, three-dimensional dynamic data isobtained either from multiple cameras or from the mathematical analysisof the obtained digital data from the light field camera system.Advantageously, the rapid variation of the light field camera eliminatesthe problem seen with moving objects that interferes with a good staticfacial recognition. Additional data can be obtained on the skin and itschanges during the physiological dynamic imaging with an infrared cameraor multispectral camera.

In addition to use in the remote laser treatment system 50′ describedabove, the dynamic imaging system described herein may also be used forother applications, such as for security system applications, use inHoloLens applications, use in telemedicine applications (e.g.,tele-imaging or telediagnostic systems), and use in other patientapplications, such as looking at the growth of various lesions over timeand aiding with diagnosis and differential diagnosis. Other applicationsfor the dynamic imaging system described herein may include specificapplications involving hospitals for patient security and operating roomapplications, customs department, airports, the state department, police(e.g., for investigative analysis of the crowd or identification aperson), the military, FBI, CIA, various other governmental agencies,and banking institutions (i.e., for account and ATM identification). Thedynamic imaging algorithm may also be used in robots, drones, inagriculture applications, or in military applications. The dynamicimaging system may also be useful at stadiums of competitive sportingevents, such as football, soccer, basketball, and hockey stadiums, andat other venues involving large gatherings of people for political ornon-political causes, which often require some permission to guaranteethe privacy and safety of the people present at the event. In addition,the dynamic imaging system may be used in smartphones for remoterecognition, and in home security systems, etc.

In one or more embodiments, the dynamic facial recognition systemdescribed herein may replace previous verification/identificationsystems used in personal life, such as passwords IDs, PINs, smart cards,etc. The dynamic facial recognition system also has unlimitedapplications in personal security, identification, passports, driverlicenses, home security systems, automated identity verification at theairports, border patrol, law enforcement, video surveillance,investigation, operating systems, online banking, railway systems, damscontrol, medical records, all medical imaging systems, and video systemsused during surgery or surgical photography to prevent mistakes in theoperating room (e.g., mistaking one patient for another one, or oneextremity for the other). The dynamic imaging system described hereinmay also be used for comparative image analysis and recognizing thetrends in patients during follow up analyses and outcome prediction. Thedynamic imaging system described herein may also be used with otherimaging modalities, such as X-ray, CT-scan, MRI, positron, photoacoustictechnology and imaging, ultrasound imaging, etc. Further, the dynamicimaging system may be used for image and data comparison of close orremotely located objects, mass surveillance to document time relatedchanges in the image, and/or recognizing a potential event and itstrend. Laser-Imaging System with Global Positioning System (GPS)Capabilities

In another further embodiment, the laser-imaging system 50′ may beprovided with GPS-based identification of the laser treatmentapplication site. With reference to FIG. 17, it can be seen that thelocal control systems disposed at local sites 51 a′, 51 b′, 51 c′ mayeach include a Global Positioning System (GPS) receiver unit 76 thatcommunicates with a cluster of GPS satellites 77. In particular, the GPSreceiver unit 76 may receive radio signals from a cluster of satellitesin orbit around the earth so as to be capable of automaticallyindicating the locations of the local control systems disposed at eachof the local sites 51 a′, 51 b′, 51 c′. As such, the specific locationof each local control system is able to be communicated to the commandcomputer 59 b of the central control system 58 via the Internet-basedconnection 60. In addition, as shown in FIG. 17, the central controlsystem 58 may also include a GPS module 78 that communicates with thecluster of GPS satellites 77 and the GPS receiver units 76 at the localsites 51 a′, 51 b′, 51 c′.

In the illustrative embodiment, as shown in FIG. 17, the GlobalPositioning System (GPS) is included in the laser delivery system (i.e.,at local sites 51 a′, 51 b′, 51 c′) located away from the control siteor doctor's office (i.e. at the remote site), providing informationlogged in its memory on where the laser system is located at any time orwhere the system might have been transported from the time of theconstruction to where it is moved in real time or is recorded in thesystem and can be retrieved.

In one or more embodiments, smartphones, computers, or the remotecontrolled laser systems at local sites 51 a′, 51 b′, 51 c′ are equippedwith GPS so that the system starts sending the information on theposition of the device as soon as the laser delivery system, a phone, ora computer is turned on.

In one or more embodiments, the GPS capabilities of the system areautomatically switched on so that when the system sends a signal, it isreceived by the laser control system along with photographs or video ifthe 3-D camera is activated at the doctor's office located away from thelaser system. When the system is turned on, the system automaticallytransmits a signal about its position at all times, which can bereceived by the control system while a data pusher tracks the device.

In one embodiment, the high definition system permits you to zoom intothe video and maintain the clarity of the image at any point.

In one or more embodiments, the laser delivery system or a smartphone ora computer may contain covert GPS trackers and the camera of the laserdelivery system identifies the patient, person, or people that arepresent along with the patient's medical history, or any other personalrecord, etc.

In one or more embodiments, the unit is programmed so that if the GPS istouched, or in any way manipulated, it will self-destruct (e.g., aftereither being exposed to the light, or mechanically handled). Forexample, piezoelectric nanoparticles may be used to create an electricalsignal that is deadly to the function of the GPS and render it incapableof sending a signal that is able to be recognized.

In one embodiment, the receiver's computer at the doctor's office firstindicates the position of the sender, then it evaluates and matchesparameters that indicate the identity of the location, by automaticallyturning on the cameras of the laser delivery system recording andsending out live stream of video as is the case, for example, with theSkype™ system. The system captures images and/or sound related to theenvironment, person(s), etc. and analyzes them using familiar facesand/or voices, such as the nurse's face and/or voice, and the necessaryphysiological functions and their changes (e.g., changes in the face,etc.) indicating the familiar person and the prior history.

The system may also send a distress signal, which initiates a remotesignal to the desired original place and automatically activates asignal to damage the laser delivery system and its software to preventits operation.

In one embodiment, the person or patient who is being examined ortreated is checked and his or her identity is verified before anycommunication between the remote system and the doctor's office isestablished by the data obtained and the software available at thedoctor's location prior to the system acknowledging that the connectionline is open to further interaction.

In one embodiment, the software at the doctor's location decides aboutwhether to permit the patient's site to communicate with the doctor,etc. by comparing or matching the present data with the previous datathat is available to the system.

In one embodiment, the doctor initiates communication by checking anyvariations and the trends in the changes of the physiological functionof the patient located at the remote area while the system evaluates andcorrelates it with available data.

In one embodiment, the remote laser delivery system always comprises acomponent that is in a permanently “on” position so as to connect thesystem to the GPS, which indicates the exact and detailed presentlocation of the remotely located system via a live GPS map, whilerecording automatically the time and the map if the unit has been moved.The GPS module 78 indicates any changes in the position of the unit onthe background map and/or generates images or other information for thedoctor.

In one embodiment, the camera system is turned on permanently andrecords the surroundings using its pattern recognition and/or 3-Ddisplay. As such, the system identifies the location and a person usingthe static and non-static or physiological changes that have occurred toidentify the person or persons or patient(s), etc., and transmits thedata to the doctor's site automatically using a 3D (stereo) camera witha head-up display.

In one embodiment, the camera at the remote laser delivery site, is adigital light field photography (DLFP) camera requiring microlenses thatcaptures the information about the direction of the incoming light raysor a plenoptic unit where the arrays of photosensors pixels are placedbehind the microlenses, and algorithms of the software are used toprocess the information obtained, as known in the art.

In one embodiment, the camera of the remote laser delivery system isequipped with a light field camera, such as Illum, CAFADIS, or roboticeye, that can provide information at different distances from the cameralocated in front or behind the focal point of the camera using softwareof the computer creating 3-D images, or stereo images using the softwareor graphics processing units (GPUs) or field programmable gate arrays(FPGAs).

In one embodiment, the doctor permits the system to analyze the data foridentification using the system software analyzing in the memory bankthe different normal versus abnormal variables, positive or negativevariables, and other associations so as to create a pattern ofsignificant association and create a coated color intelligentassociation of the past and present images.

In one embodiment, after verification of the person, the system permitsexamination of a lesion, surface, or internal structure with ultraviolet(UV) to infrared light, optical coherence tomography (OCT), ultrasound,or photoacoustic images to compare with the existing information fromthe patient and from other patients so as to provide potential diagnosisand options for therapy, while the system verifies the location,person(s) or patient(s) involved, etc. before proceeding with thetherapy or an action and before the doctor runs the simulation of alaser treatment, if needed.

In one embodiment, the system provides for remote imaging or patternrecognition for a patient, persons, a surface, mucosal marking, orlesion along with verification using multi-spectral imaging, OCT,ultrasound or photoacoustic imaging and physical and physiologicalanalysis to compare the surface lesions for shape pattern circulationthickness in 2 to 3 dimensions with the lesions or faces seen previouslyand existing in its file to diagnose a lesion, face, etc. and analyzethe status of the progression of a disease, aging, and provide a list ofoptions to be used in each case that can be complemented with theexisting worldwide experiences stored in the memory of the computer ofthe unit.

In one embodiment, the connection to another requested computer isrouted through the central location to verify the signal as a “friend”with all the patient's information or person's recognition live prior togiving the permission to be connected to another system elsewhere.

In one embodiment, all the process and personal recognition isautomatically stored at the central location and may be retrieved againby a doctor or a person already known by the system. The desired 2-D,3-D, or video images may be retrieved along with recordings of theconversations between the parties.

In one embodiment, the control system performs only one way recognitionof the person at the remote location. In another embodiment, for anextreme security sensitive and urgent communication, the control systemverifies the people or each person on both sides live from its existingfiles, and then opens the communication to establish a secure bilateralsystem, mostly for short and concise communications while the two-waysystem can initiate only from the doctor's office side, thereby addinganother layer of confidence or veracity to the information.

In one embodiment, the existence of the system is known to only thepeople with top security clearance that carry a specific computer andcell phone.

In one embodiment, the computer of the laser delivery system at both thecentral and the remote location, or the laptop, and cell phones arereplaced with new ones after the entire information is retrieved and thedata is cleaned.

Laser-Imaging System with Photodynamic Therapy (PDT) Capabilities

In another further embodiment, the laser-imaging system 50, 50′ may beconfigured to provide remote photodynamic therapy (PDT) for the patient.More particularly, in this further embodiment, the treatment laser ofthe laser generation system may be configured and arranged to performthe photodynamic therapy (PDT). In this embodiment, the PDT treatmentmethod utilizes the combined effects of a photosensitizer in presence ofoxygen and a light of a specific wavelength emitted by the treatmentlaser that is absorbed by the photosensitizer so as to create a chemicalreaction, thereby producing singlet oxygen and reactive species. Theseare in oxidative processes and are toxic to the tissue damaging thecellular mitochondria if the photosensitizer is located inside the cell.If the photosensitizer is located outside the cell in contact with thecell membrane, it damages the cell membrane (e.g., endothelial cellmembrane of the vessels). The damage to the endothelial cell wall of thevessels causes platelet aggregation, cloth formation, and closure of thevessel. Addition of an anti-vascular growth factor to the PDT enhancesdamage to the abnormal blood vessels produced by the tumor.

In this further embodiment, after inducing the physiological changesdescribed above and verifying the patient's identification for theprocedure, the site and location of the intravenous injection of a PDTphotosensitizer and/or a surface lesion is captured by the imagerecognition sensor 73 of the laser-imaging system 50′ and displayed tothe remote physician on the 3D display so that it can be verified by thephysician. Then, a combination of the photosensitizer and anti-VEGF,such as Avastin or other agents (e.g., anti-platelet derived growthfactors) is applied to enhance damage to a cancerous lesion, such asskin or mucosa. Also, in addition to the photosensitizer, an anti-cancermedication, an anti-cancer immunotherapy medication, such as Braf andMek targeted therapy, and/or a programmed cell death PD-1 immunotherapyto enhance damage to the cancerous lesion may be applied. After the PDTprocedure, the patient is advised by the physician to avoid sun exposurefor one week to prevent a sun radiation effect on other parts of thebody.

Also, in this further embodiment, after inducing the physiologicalchanges and verifying the patient's identification for the PDT procedureusing the 3D display, nanoparticles coated with a biocompatible polymerand a photosensitizer conjugated with a thermosensitive polymer, such aschitosan, etc., are applied to the cancerous lesion of the patient. Thenanoparticles may be metallic, non-metallic, synthetic, organic,non-organic, a hybrid, magnetic, paramagnetic, diamagnetic,supramagnetic, non-magnetic, and combinations thereof. The nanoparticlesmay be in the form of graphene-oxide quantum dots, graphene-zinc oxidequantum dots, graphene nanotubes, and/or carbon nanotubes. Thenanoparticles may contain a combination of two to three elements, suchas gold, gold-iron oxide, iron oxide, iron-zinc oxide, metallicnanoparticles, polylacticglycolic acid nanoparticles, ceramicnanoparticles, silica nanoparticles, silica crosslinked block polymermicelles, albumin-based nanoparticles, albumin-PEG nanoparticles,dendrimer attached magnetic or non-magnetic nanoparticles, etc. Thenanoparticles also may be in the form of dendrimers, micelles,fullerenes, quantum dots, nanoshells, nanocages, nanorods, nanotubes,nanowires, and combinations thereof. The nanoparticles may be formedfrom gold and silica, ferric oxide, ferric, cadmium sulfate, platinum,etc., and may be coated with polyethylene glycol (PEG), fatty acid,chitosan, or another biocompatible molecule and conjugated with a tumorantibody and a photosensitizer. The nanoparticles may be conjugated witha tumor antibody and a cell penetrating peptide (CPP) so as to enhancethe penetration of the photosensitizer in the tumor cells. Thephotosensitizer and/or nanoparticles, may be given intravenously,applied topically, or taken orally. Preferably, the anti-VEGF is appliedlocally to avoid the side effects of systemic administration, unless itis delivered as coated nanoparticles which are functionalized with anantibody to seek specifically a tumor cell. In general, thefunctionalized nanoparticles are taken up preferentially by the tumorcells and the growing proliferating endothelial cells of the abnormaltumor cells. When administered intravenously, the photosensitizer and/ornanoparticles may be configured to reach the desired area and attach tothe desired tumor cells by utilizing a monoclonal antibody, a polyclonalantibody, or aptamer-coated nanoparticles to seek out the tumor cells.Also, in this embodiment, functionalized pluralities of nanoparticles,which are coated with a photosensitizer, may be given topically orsystemically to a tumor located on or close to the skin or mucosa of thebody.

In this further embodiment, the nanoparticles may be in the form ofthree dimensional semiconductor devices using light or ultrasound energyto generate electrical energy to provide a photovoltaic effect. Inembodiments, the nanoparticle material may be ceramic, plastic, silicon;particles of crystalline silicon may be monocrystalline cells, poly ormulticrystalline cells, ribbon silicon having a multicrystallinestructure, nanocrystals of synthetic silicon, gallium/arsenide,cadmium/selenium, copper/indium/gallium/selenide, zinc sulfide, ironsulfide, iron-platinum, indium/gallium/phosphide, gallium arsenide,indium/gallium nitride, a nanocrystal, such as cadmium/selenium (Cd/Se)and a metal, e.g., a CdSe/Au nanometer-sized composite particle aspreviously described, particles of a variety of semiconductor/metal andsemiconductor/semiconductor hetero-junctions, e.g., particles based uponsemiconductor/metal hetero-junctions between group II-VI, IV, III-V,IV-VI, referring to groups of the periodic table, metal-oxide, ororganic semiconductors and a metal, and in particular those based uponSi/Au, GaAs/Au, InAs/Au, and PbS/Au hetero-junctions. The quantum dotsand/or semiconductor nanowires may also be biocompatible short peptidesof naturally occurring amino acids that have the optical and electronicproperties of semiconductor nano-crystals, e.g., short peptides ofphenylalanine. The particles can consist of both inorganic and organicmaterials, as previously described. In addition to being stimulated bylight and ultrasound energy, the nanoparticles may also be stimulated bymicrowave radiation, magnetic fields, alternating magnetic fields, andradiofrequency (RF) electromagnetic radiation.

Moreover, in this further embodiment, the application of the treatmentlaser for PDT is substantially applied in a continuous wave (CW) fashionat wavelengths that are absorbed by the nanoparticles and/or thephotosensitizer. The laser spot generated by the treatment laser may beapplied as a single spot covering the entire cancerous lesion andslightly beyond it. Alternatively, the treatment laser may be applied asa small spot, but using a painting or oscillatory technique bydisplacing an electrically controlled prism or a mirror, which islocated in front of the treatment laser beam, in an oscillatory manner.In this embodiment, the treatment laser of the laser generation systemmay produce a wavelength appropriate for the photosensitizer (e.g., awavelength of 405 nanometers (nm) to 420 nm, or 500 nm to 600 nm, or 635nm, a near-infrared laser from 600 to 1060 nm-1550 nm and more, up to aninfrared wavelength) which is absorbed by the photosensitizer andpenetrates the skin and mucosa up to a distance of 1 centimeter (cm) inthe tissue. In one or more embodiments, the treatment laser may comprisea powerful non-coherent light source from any light, such as a xenon ormercury lamp, rather a coherent light source.

Furthermore, in this further embodiment, the patient's cancerous lesionmay be pre-treated by the topical application of the photosensitizerand/or nanoparticles through the skin or mucosa, for a period of 1-30minutes depending on the thickness of the lesion to be treated. Thephotosensitizer may comprise one of aminolevulinic acid, methylaminolevulinate, Verteporfin, and riboflavin, etc. Advantageously,riboflavin is a non-toxic photosensitizer which still normally kills thetumor cells, and cross-links the collagen around the tumors so as tostrangulate and close their vascular supply as a result of itsphotodynamic effect. In one exemplary embodiment, a Verteporfinphotosensitizer preparation is conjugated with cell penetrating peptides(CPP) or activated cell penetrating peptides (ACPP) and polyethyleneglycol (PEG), etc. coated nanoparticles, and after the identity of thepatient is verified in the manner described above, the preparation isadministered intravenously to reach internally and externally-locatedtumor cells. In this embodiment, the size of the nanoparticles ismaintained below 10 nanometers (nm) to enhance their secretion throughthe kidney and urine. In other embodiments, the sizes of thenanoparticles are from 1 nm to 1000 nm. When the photosensitizer is anaminolevulinic acid or methyl aminolevulinate, it is preferentiallyabsorbed by the lesion specific antibody-coated nanoparticles so as toprovide a lesion specific response in the lesion because the area of theskin or mucosa that does not contain photosensitizer will not respond tothe light and will not be affected.

In this further embodiment, riboflavin may be applied topically in abiocompatible solution, the treatment laser may be configured to emitultraviolet (UV) laser light therefrom. For example, topicalapplications of 1% to 2% riboflavin in a biocompatible solution may beapplied as drops for 1 to 20 minutes depending on the concentration ofthe riboflavin and the power of ultraviolet laser light (e.g., 2 mW to20 mW) to activate the riboflavin (the higher the concentration ofriboflavin or the laser power, the less time is needed to concluderadiation). In this further embodiment, the antibody nanoparticles maybe conjugated with riboflavin and polymers, such as polyethylene glycol,chitosan, etc. for systemic, intravenous, or local application.

Also, in this further embodiment, the photosensitizer (e.g., riboflavin)may be used topically in physiological solution to the infected skin ormucosal wound by bacteria, fungi, or other organisms, such as amoebas,that are therapy resistant to medication, alone or with antibiotics orantifungals or the medication, while the infected wound is exposed to UVradiation of 380 mm in wavelength or higher for the appropriate timedepending on the riboflavin concentration and/or the laser energy tocross-link the collagen of the tissue and simultaneously kill theoffending organism. Riboflavin is vitamin B2, and is important formaintenance of the health of the human body. In this further embodiment,the laser wavelength may be chosen depending on the absorption of thephotosensitizer (e.g., the absorption of the riboflavin).

In addition, in this further embodiment, after inducing thephysiological changes and verifying the patient's identification for thePDT procedure using the 3D display, the presence of any lesion on theskin or mucosa, and the size, elevation and the physiological parametersof the lesion are compared before, during, and/or after the procedurewith the results of the previous initial examination. In thisembodiment, the surface of the skin or mucosa is treated with a solutioncontaining pluralities of coated, thermosensitive nanoparticlesconjugated with a polyethylene glycol (PEG) and a thermosensitivepolymer, such as chitosan, a cell penetrating peptide (CPP), aphotosensitizer, and at least one more medication that is released fromthe chitosan, when the temperature of the nanoparticles reaches 41 to 43deg. C. In this embodiment, the temperature of the lesion may bemaintained at 41 to 43 deg. C. to release the medication from thethermosensitive polymer and to prevent a pain sensation. The temperaturemay be raised to 45 to 47 degrees C. for a very short time, whileapplying prophylactically, a topical anesthetic medication in order tokill the cells without creating tissue burn. In another embodiment, thephotosensitizer may be released from the thermosensitive polymer coatingof the nanoparticles when the temperature reaches 38 to 43 deg. C. Also,in another embodiment, the temperature may be raised up to 43 to 45degrees C., 45 to 47 degrees C., or up to 50 deg. C. to kill thecancerous cells of a lesion on the skin mucosa and other accessibleareas. The treatment laser of the laser-imaging system 50, 50′ not onlyactivates the thermosensitive photosensitizers, but also heats up thenanoparticles, etc. Any follow-up treatments are done with the sameremotely-controlled laser and camera system which is equipped with 3Ddisplay described above for accurate recognition and examination of thelesion.

In this further embodiment, when the patient has a skin or mucosallesion or cancer, the combination of the photoacoustic components of thesystem (i.e., the laser and ultrasonic transducer described above)creates not only a 2D image of the surface structure, but using thesystem as photoacoustic tomography provides a 3D image of the lesionproviding gross or microscopic information about the depth that thelesion has penetrated inside the skin or mucosa, while also providingvolumetric information to be measured in the follow-up examination ofthe patient. The 3D image of the lesion may also provide informationabout the effect of the treatment on the lesion in the subsequentfollow-up period, while the data is simultaneously transmitted to adoctor or observer located at the remote site. Also, the photoacousticsystem may be used to provide information about a small region of alarger structure, such as the breast, exhibiting early signs ofpotential cancer in the tissue in an early stage of the disease when thelesion is very small (e.g., 1 to 2 mm dimension) that is difficult toimage by using electromagnetic radiation (i.e., x-ray) or computerizedtomography (CT) scan, etc.

Also, in this further embodiment, the photosensitizer and/ornanoparticles are treated with a laser, for a period of 1 to 30 minutesor more, under observation by processor control, while the laserdelivery system is under the control of the photoacoustic system tocontrol the amount of energy delivered to the lesion for a predeterminedtime. A thermoacoustic response is recorded simultaneously and a thermalmap is created continuously, to prevent heating the tissue beyond thepredetermined temperature of between 39 and 44-47 deg. C. as needed,etc. without creating a thermal burn.

In this further embodiment, the treatment system provides a combinationeffect of PDT if the temperature is maintained below 42 degrees C., andsimultaneously a thermal effect if the temperature is gradually raisedusing a photoacoustic imaging system to keep the temperature below thetemperature in which protein denatures, which is about 50 degrees C.depending on the duration of the application and power of the treatmentlaser.

In this further embodiment, the treatment laser of theremotely-controlled laser system is used for the PDT of a Basal cellcarcinoma, squamatous cell cancer of the skin, angioma, senilekeratosis, precancerous melanosis, melanoma, cancer of the mouth mucosa,throat, nose, vaginal mucosa, cervix, or the uterus. The PDT may also beused to treat photoaging, acne vulgaris, dry eye to enhance meibomiangland secretion, Bowen's disease of the eye or lids, and the PDT mayalso be used to deliver strong light therapy to damage and killbacteria, such as Propionibacterium acnes, bacteria in blepharitis, orto treat infected corneas that are infected by bacteria, fungi, viruses,and amoebas, etc.

System Software Platform

The software system that is employed to control the laser procedure(s)with the laser-imaging systems of the invention includes client/serverarchitecture and a TCP/IP communication protocol. The client/serverarchitecture comprises a computer science paradigm, where clients andservers are deemed separate software processes that can run on the sameor different computers.

In some embodiments, the software recognizes the computer at thesurgeon's site as a client, while the local control module at the remotepatient's site is recognized as a server.

According to the invention, communication by and between the remotecontrol module 64 and the local control module 62 is facilitated via webservices implemented in .NET remoting technology.

When a physician at a remote site sends a control or data acquisitioncommand, the command is first transmitted to the local control module 62(or server) through the .NET remoting interface. In response to thecommand, the local control module 62 controls hardware compartmentsthrough a hardware specific interface (e.g., RS-232C interface, parallelcommunication protocol).

The communication speed between the client and the server will depend onseveral factors such as 1) the distances between the client and theserver, 2) network traffic conditions, and 3) the size of data (e.g.,images) being transmitted.

As will readily be appreciated by one having ordinary skill in the art,the present invention provides numerous advantages compared to prior artmethods and systems for laser coagulation procedures. Among theadvantages are the following:

-   -   The provision of laser-imaging systems, which will significantly        reduce laser transmission and, hence, procedure time. For        example, the length of time for the laser photo-coagulation        treatment for diabetic retinopathy will be reduced from 30-60        minutes per procedure to only two minutes. In general, the        duration of a single laser pulse, plus the time it takes to        perform the subsequent laser application multiplied by the total        number of pulses required is equal to the overall procedure        time. The manner in which the pulses are applied in a        conventional contact system (e.g., using a contact lens        positioned on the cornea in order to see the fundus) requires        the physician (i.e., ophthalmologist) to perform the procedure        laser shot by laser shot (i.e., to place a single laser spot at        a desired location and then move on to the next location). When        performing the procedure on a patient that requires 1000-2000        laser spots, the laser coagulation procedure can easily take 30        minutes or more. The laser coagulation system disclosed herein        uses millisecond laser pulses (and in some cases, less than        millisecond pulses), and each subsequent laser shot is performed        automatically, in some embodiments, by utilizing an oscillating        mirror (e.g., oscillating mirror 220 described above) placed in        the path of the laser beam such that the laser beam is displaced        without requiring any manual means. In addition, the wide angle        camera employed in one or more embodiments described herein        enables the entire retina to be viewed, rather than just a small        portion thereof, thereby further reducing procedure time. As a        result of these enhanced features, the laser coagulation system        described herein substantially reduces the overall procedure        time. In one embodiment, the remote operation module of the        laser coagulation system is configured to perform a fully        automated and continuous laser coagulation procedure over the        entire area of the retina in a period of time no greater than        approximately 2 minutes (or no greater than 2 minutes) in an        actual control mode.    -   The provision of laser-imaging systems, which will also reduce        the probability of error associated with manual surgery (tremors        and misjudgments) via a more precise computerized control        mechanism, with additional fail-safe features, and a wider angle        imaging camera for retina diseases. This offers more choices for        various lasers with different wavelengths, intensities, and        action than was previously possible.    -   The provision of laser-imaging systems, which will also allow a        physician or surgeon to perform a procedure at a remote site,        via a high-speed reliable Internet® connection, thus eliminating        the need for the patient to travel a long distance to be treated        at a specialist's office or, in the case of military field care        or space exploration units, allowing patients to be treated        immediately on-site.    -   The laser coagulation system described herein is in the form of        a non-contact system that does not require the use of a contact        lens or any other device in contact with the eye of the patient.        The laser coagulation system embodied herein also does not        require the physician to indent any portion of the patient's eye        (i.e., no indenter or scleral depressor is required to indent        the peripheral portion of the patient's eye). As a result, the        patient is far more comfortable during the laser coagulation        procedure.

The cameras employed in the systems can also be equipped withappropriate diode lasers and filters for auto-fluorescence photographyand angiography of the retina.

One can also develop a miniature version of the system by using printingtechnology, such as inkjet, to generate and integrate the micro-opticalparts not only on hard substrates, but also on those that are flexible.Having a miniature or micro-system will allow further use of suchtechnology in hard to reach places.

In one embodiment, dynamic facial recognition (DFR) is complementary toartificial intelligence (AI) for differentiating dynamic changes thatcan occur in a short period of time or over a long period of time forthe identifying, quantifying, and/or characterizing a disease conditionin a subject or in an organ of the subject and define the precise amountof change.

In one embodiment, a combination of dynamic facial recognition (DFR) iscomplementary to AI or essential (e.g., recognizing a patient insecurity systems, especially in health and patient care maintainingconfidentiality of each patient and its disease process). AI and DFR maybe used with or without a bot (i.e., a software application forautomating a certain task) to simplify confirmation of a patient by manydifferent means which are not easy to reproduce, such as simultaneousthermometry with infrared (IR) imaging as known in the art or changes ofa surface due to touching a tissue by an instrument during the surgery,storing the data so that only the patient and his or her verified doctorcan access it, etc.

In one embodiment, the use of Metaverse, which is a digital combinationof virtual reality (VR), mixed reality (MR), augmented reality (AR), towhich facial recognition technology and AI is added in order to closethe circle of security in the medical field permitting live interactionbetween the doctor and patient without worrying about the breach ofconfidentiality and security of the patient's data which can bepresented along with the patient's data, or needed images obtained by acamera, CT-scan, x-ray, MRI, photography or optical coherent tomographyor by hyperspectral imaging etc. The system not only simplifies thecommunication between two or more parties, but takes less time since thesystem software provides immediately the needed information from thepast session, diagnosing improvement or worsening of a condition (e.g.,an inflammatory process, a skin lesion, or a benign or malignant lesionin the body etc.)

In one embodiment, DFR, AI and Metaverse used with a fluidic lens camera(U.S. Pat. No. 9,016,860, which is incorporated by reference herein inits entirety) or any other camera can be provided with or without a bot,to obtain within a few second multiple information, such as the name,patient's disease, medications, or family history almost simultaneously,such as patient's recognition, the visual aberration of the eye, correctthe aberration, provide a prescription, while the OCT system attached tothe camera can diagnose a normal cornea versus a keratoconus by itssteep image, evaluate the status of a cataract measuring the density ofthe lens substance and grade it, presence or absence of diabeticretinopathy by abnormal microaneurysms of the retinal vessels, bleeding,or changes in the thickness of the retinal layers or presence or absenceof the sub-retinal fluid, or presence or absence of the age relatedmacular degeneration by loss of retinal pigment epithelium or reflectingdrusen under the RPE, or presence or absence of abnormal or leakycapillaries or the density of the retinal capillaries, or loss ofretinal capillaries or to run a pressure test (U.S. Pat. Nos. 10,736,571and 7,828,440, which are incorporated by reference herein in theirentireties) to evaluate retinal oxygenation and recovery of thecirculation or loss of the retinal ganglion cells around the optic nervehead or loss of peripapillary capillaries, or central retina orperipheral retina in glaucoma, diabetic, presence or absence of a tumor,presence or absence of a retinal detachment, status of the retinalganglion cells, inner and outer retinal nuclear layer, status of thephotoreceptors of the retina, sub-retina fluid, intra-retinal fluid anddifferentiate it from a retinal or sub-retinal hemorrhage, status of theage related macular degeneration etc., all done in less than one tenseconds, saving patients and doctors time and even providing suggestionsfor the follow up and treatment to a doctor to decide and confirm orprovide new medication(s) while maintaining the patient security.

In one embodiment, dynamic identity recognition utilizes components ofartificial intelligence including training, validation with images anddata, combined with a technical network, such as a convolutional neuralnetwork, the training include recognition of obvious characteristics,such as the face, eyes, nose, mouth, etc. requiring a focused image in2-3 D format or dynamically induced changes in the image that provideadditional data along with voice recognition. In one embodiment, the DFRand AI can be interactive when equipped with a bot (robot) to performsimple communication with the patient, such as initial instruction tothe person and/or a patient (e.g., to smile please, or frown pleaseetc.), which initiate s the dynamic response in the person's mouth oreyes' surrounding structures used to simplify the computer work inrecognizing the changes and direction of changes that has occurredeliminating the need for an instructor to be present and the unitsimultaneously analyzes the statics areas and dynamic facial changes andit provides for rapid computer-assisted DFR and the bot recognizes thevoice.

In one embodiment of the DFR system, the bots assist with the patient'shistory, family history, parents and grandparent's history of a disease.

In one embodiment, a DFR system along with a bot (e.g., a chat bot, suchas Amazon Alexa®) is used to simplify the work of AI by creating asystem positioned between the patient and the machine where a bot ormultiple bots ask one or more questions from the patient or a person andgets a response. Exemplary questions are: (i) “What is your name?” (ii)“Smoking (yes or no)?”, (iii) “High blood pressure and/or diabetes (yesor no)?” Answers to these questions are recoded and provide a directionor a shortcut for the AI which normally requires a large number ofnormal references (images, voice, or video, etc.) from a large number ofsubjects, with or without a disease, and experts in data collection toobtain a response, while the bot can rapidly categorize the patient'sproblem by the patient's response leading to a more refined and shorterroute time, with or without the patient asking a question or hearing aresponse from the bot for AI to provide a faster diagnostic and/ortherapeutic response.

In one embodiment, the combination of DFR, AI, Metaverse, and/or a botused with a laptop or smartphone may ask a verbal or written questionrelated to any situation or field of interest that one would like to geta response, such as in security systems, medical fields, telemedicine,in the financial field, banking, ATM, etc. Human or animal issues or anyspecific or non-specific disease processes along with one or more imagestaken over a period of time or genetic diseases, or trauma or any imagetaken by ultrasound, X-ray, CT scan, MRI of any body's part, skin ormucosa, and/or internal and external organs with or without contrastagents, etc. external images or internal images or images of an objects,living or not living or images obtained by hyperspectral camera, 2-D or3-D images or IR images, visible light images or ophthalmic images orretinal images or images of head and neck, chest, breast, abdomen,intestine, lung, head, pelvis, bone and muscles, and/or extremities of ahuman or animal, macroscopic or microscopic images or video images, etc.A bot might be able to communicate using various languages if initiatedby the person.

In one embodiment, in the medical field or telemedicine, DFR with orwithout a bot and its algorithm or another camera, is combined withMetaverse, etc. and a third information obtained from a bot tocategorize the patient (or a person) where the bot asks a question “Whatdisease(s) you have had?”, and the patient's response rapidlycategorizes the captured images (e.g., one or more diseases, such asdiabetes or hypertension or heart, kidney, lung, or breast issues, CNS,eye, musculoskeletal, or intestinal tract or genitourinary tract or skindisease or an infection) that will be recorded and becomes useful todirect the system toward the manifestation of the disease (e.g., if thepatient indicates diabetes and a retinal photograph is presented to thesystem, the system can rapidly differentiate and categorize easily animage of a retina from a diabetic patient from an image of a retinahaving a hypertensive disease or from a retina having age relatedmacular degeneration, etc.) and rapidly diagnose it without having tosearch millions of other unrelated photos, diagnose and stage thedisease process or the change from previously existing image. In thisbot-assisted DFR and imaging for a disease process with or without AI,the diagnosis is simplified or the computer takes a short cut if thepatient provides an identifier word, eliminating time consumingcommination specially if the disease is known ahead of time. Thisprocess DFR, bot, and person's voice response simplifies rapidprocessing and recognition in medical patients and doctor telemedicineor at the airport or sport stadium or in the army, or at any place wherelarge groups of people have to be checked in or out. In a securitysituation, the voice recognition and the patient's name are robotic (bya bot) along with DFR, the system can screen rapidly patient(s) or thepeople, etc.

In one embodiment, the camera is a combination of an automated phoropteras described by Peyman (U.S. Pat. No. 9,191,568, which is incorporatedby reference herein in its entirety) is equipped with deep learningsystem, dynamic facial recognition, with or without bot and Metaversefor automated correction of refractive power and automated in focusphotography using Shack-Hartmann sensor with convolutional neuralnetworks (CNN), and trained CNN for detection of the diabeticretinopathy, or vision threatening diabetic retinopathy, dry or wet agerelated macular degeneration, optic nerve changes in glaucoma opticnerve head, or the changes in corneal thickness and curvature,conjunctiva inflammation, lid margin blepharitis, iris atrophy,pupillography, lens, cataracts, and the optic nerve cup and retinallayers using either an OCT, or ultra-wide field scanning laserophthalmoscope or hyperspectral or multispectral camera, simple lightphotography or confocal microscopy for corneal diseases and or cornealtopography for keratoconus, etc.

In one embodiment, in a patient suspected of having an early or lateAlzheimer's disease, the visual disability due to refractive errorinterferes with the patient's visual ability to read a chart visualdisplay, which requires correction, under normal with the standardphoropter, requiring the patient's cooperation and intelligence torecognize between two sequential images presented to the patient, thisvisual disability caused by a disease such as age related maculardegeneration interferes with the patients' disability and potentially byAlzheimer's disease that can produce the same results (i.e., not beingable to recognize images properly). In one embodiment, the camera is acombination of an automated phoropter as described by Peyman (U.S. Pat.No. 9,191,568) and is equipped with deep learning system, dynamic facialrecognition, with or without a bot and/or Metaverse with convolutionalneural networks (CNN), and trained CNN, for automated correction ofrefractive power and automated in focus photography of the retina by itsShack-Hartmann sensor and fluidic lenses to automatically correctrefractive error of the eye while the system is equipped with a visualdisplay for the patient to see and read various images combined withletters and numbers of various sizes. Initially, in the right side upposition first, the patient is requested to read and recognize theimages, and this is recorded. This is followed with upside down positionof the same visual display to read, this will record the degree ofpatient's cognitive ability to read while the refractive power isautomatically corrected by the system without the need for standardphoropter which requires a trained assistant to ask: “Is this better, orthis?” The above system corrects automatically the refractive errorswith its fluidic lenses and its Shack-Hartmann sensor without the needfor the patient's input to eliminate the effect of functional ability ofthe patient, caused by refractive error from his or her ability torecognize letters, objects or numbers in an right side up position andverbalize it, then the inverted position of the same visual display isdisplayed inside the unit, thereby indicating objectively the degree ofcognitive ability of the patient, while any other organic diseases suchas corneal opacity, or cataracts and or retinal degeneration can beautomatically ruled out by the deep learning (DL), CNN, Metaverse, etc.and the privacy of the exam is guaranteed by the DFR, when the patientis repeatedly examined for improvement of the condition, such ascataract removal or medication used by the doctor to treat the retinaldiseases or the Alzheimer's disease.

Since the Alzheimer's disease occurs in an age that coincides with otherdiseases such as age related macular degeneration or the loss ofhearing, it has not been easy to test the patients, e.g., with readingcharts since the patients' refractive errors need to be corrected priorto reading a chart or seeing an image to test their recognition ability.For correction of refractive error, if is done by a standard phoropter,the patient has to differentiate subjectively between various lensesplaced in front of their eyes. This process is not easy even for thenormal patient and can be time consuming. The subjective correction ofthe visual aberration is often not reliable in patients in early stageAlzheimer's disease. In addition the diagnosis of the age relatedmacular degeneration requires an ophthalmologist to look inside the eyeto see the status of the retina or macular area. Fundus photography,using standard fundus camera or optical coherence tomography or sweptsource scanning laser ophthalmoscope requires also presence of anophthalmologist to make the diagnosis. Therefore, the conventional testsare too subjective to be useful for early diagnoses of Alzheimerdisease.

In one embodiment, one uses an automated phoropter and camera thatautomatically corrects refractive error of a person in both eyes inabout 2-5 seconds and immediately takes a photograph of the centralretina using a Shack-Hartmann sensor and fluidic lenses to correct therefractive error without the need of the patient's input and using aphotograph or a OCT 2-3D image of the retina can diagnose automaticallypresence of absence of an age related macular degeneration with AI, aneural network (NN) or convolutional neural network (CNN) or Metaverseor machine vision and a bot that collects and responds to a voice thatis analyzed with dynamic facial recognition software, differentiatingage related macular degeneration from a macular hole, or diabeticretinopathy or central or branch vein occlusion or other diseases, whilesimultaneously uses its visual display (used as reading) which isreplaced automatically with a software with a cognitive or Alzheimer'schart 700, 700′ (see FIGS. 27 and 28) with visual acuity 712 having aseries of numbers, letters of alphabet or mingled drawings of objectspartially superimposed on the top of the each other and one or morephotographs of famous people such as president or vice president of acountry such as USA, etc. for the patient to recognize and record theresponse by the unit's software and AI to recognize correct and wrongscore of the patient's response and in a second stage, the same image isinverted and the patient reads the numbers etc. and the information iscollected by the bot and scored and percentage of correct responsesversus all correction are mathematically analyzed, e.g. providing ascore or <60-160 where <60 represents very low or poor recognitionand >100 as normal or good depending on the age of the patient.

In one exemplary embodiment, the correct first test provides a certaincorrect cognitive score (e.g., total of 40) and the second invertedimage provide a higher correct cognitive score with higher value (total60 score), the total number correct cognitive scores can be added toreach a total number which also can be expressed in percentage (notethat many variations of this test can be developed for each conditions,e.g., for age related development of a child or a person.

In one embodiment, the test is performed before and after a therapeuticprocedure, e.g., a cataract surgery is performed or before or afteradministration of a medication affecting the patient's clarity of mindand cognitive ability for recognizing the letters, etc. offered by thetest.

In one embodiment, the aforedescribed Alzheimer's and the cognitive testmay be used for many other situations (e.g., in children, adolescents,Parkinson's disease patients, before and after traumatic head injuries,such as those sustained in boxing, football or other sports, afteraccidents with head trauma, or a disease involving the brain, rupture ofaneurysm, a disease process, such as virus infection COVID and longCOVID or brain fog bacterial, fungal, viral, or parasitic braininvolvement of a brain tumor or a medication, etc.).

With reference to FIGS. 27 and 28, the numbers 710 in the first line 702should be sufficiently large to the person so that the person can readit without much difficulty even if they have some retinal problem. Inthe second line 704 of FIGS. 27 and 28, the figures depict from left toright, a glass 714, a wheel 716, a candle holder with a burning candle718, and a clock 720 with two arms showing the time (without thenumbers) where the time 11:10 is depicted. In the third line 706 ofFIGS. 27 and 28, the figures depict a graphical representation of twoU.S. presidents and a first lady 722 (e.g., President George W. Bush,Michelle Obama, and President Barack Obama). In the fourth line 708 ofFIGS. 27 and 28, the figures depict three animal pictures 724, 726, and728. In FIG. 27, the first picture is right side up, and in FIG. 28, thepicture is upside down, and the person will have more difficulty inrecognizing the objects depicted in FIG. 28. In one or more embodiments,each line section 702, 704, 706, and 708 has a score that isautomatically recorded as correct or wrong. The total computed scoreshows the percentage of a person's ability to recognize an object, thesecond figure (i.e., FIG. 28) also tests if the person is able to keepthe result of the first picture in his or her memory (short term memory)versus identification of the presidents (long term memory).

In one embodiment, during the administration of the cognitive testdepicted in FIGS. 27 and 28, in the first row or line 702, the patientlooks at the number and within one second the fluidic photopter focuseson the visual display to correct the refractive error of the eyes withthe shack-Hartmann System and fluidic lenses. This unit may alsoautomatically takes a picture from the retina for AI, DFR, and NN andCNN, and Metaverse, etc. In the first row or line 702 of FIGS. 27 and28, the patient's ability to read the numbers is measured. In the secondrow or line 704 of FIGS. 27 and 28, overlapping objects 714, 716, 718,and 720 are used to analyze the brain ability of patient todifferentiate between the different objects 714, 716, 718, and 720. Inthe third row or line 706 of FIGS. 27 and 28, the system checks longterm memory of the patient by determining if the patient is able toidentify the former U.S. presidents and first lady. In the fourth row orline 708 of FIGS. 27 and 28, the patient is asked to identify variousanimals, and his or her answers are recorded by the bot as he or shedictates the answers. The inverted images in FIG. 28 check the shortterm memory of the patient, since the patient just read and recognizedthe images of FIG. 27. However, it requires more work by the brain toremember them and interpret the inverted images. All responses arerecorded by the bot system and analyzed, thereby providing a score from100% correct (all responses correct) to a lower percentage of correctresponses, which showing the severity of damage to the recognitionability of the patient.

In one embodiment, an imaging system having a dynamic facial recognition(DFR) imaging and an algorithm is added to AI software fordifferentiating dynamic changes that can occur in a short period of timeor over a long period of time using DFR to identify, quantify, and/orcharacterize a disease condition in a subject or in an organ and definethe route to its treatment.

In one embodiment, an imaging system having combination dynamic facialrecognition (DFR) is complementary to AI or essential to AI forelectronically recognizing a person or patient remotely in securitysystems, specially in the health sector and patient care to maintainconfidentiality of each patient, his or her medical records, diseaseprocesses, etc. where AI and DFR simplifies, and is less time consuming,for identity confirmation of a patient by many different means which arenot easy to reproduce such as pixelated facial images and its dynamicchanges of wrinkles by frowning or smiling, etc. using a subtractionalgorithm to specify the changes rapidly for facial recognition of or,in addition to person's voice recognition, to enhance the process ofcomputer recognition, in security systems or maintain confidentialityand/or combined with simultaneous live thermometry imaging or change ofa tissue surface due to touching it with an instrument during, e.g., asurgery, storing the data so that only the patient and his or herverified doctor can access it, while an artificial intelligence tool canevaluate images of tissue to identify and quantify areas of change,e.g., due to inflammation, degeneration infection, or a tumor, etc. anddiagnose the disease process and recommend management by accessing thedata in its software, or the system may be used for industrial use forquality management, or by a doctor for use in a short period of time.

In one embodiment, an imaging system with or without a bot having theuse of Metaverse, or Eon Reality in teaching or a digital combination ofvirtual reality (VR), mixed reality (MR), and/or augmented reality (AR)combined with dynamic facial recognition technology and AI to close thecircle of security in the medical field or in the security business,permitting live virtual interaction between the doctor and patient orparties without worrying about the breach of confidentiality andsecurity of the patient's data, which can be verified by the unit'ssoftware and presented along with the patient's images obtained by acamera, or is made available images of a patient's CT-scan, X-ray, MRI,photography or optical coherent tomography (OCT) or images obtained byhyperspectral camera, or light field camera, etc. The system not onlysimplifies the communication between two or more parties, but is lesstime consuming, since the system's software provides immediately theneeded information (e.g., change from the previous image) from the pastpatient's visit, differentiating improvement or worsening of a conditionby DFR software and AI (e.g., an inflammatory or degenerative process, askin lesion, or a benign or malignant lesion in the body, etc. presentedto the doctor, or the discrepancy between a product from the originallydesigned concept, for eliminating human mistake in industry, banking, orin quality management etc. presented to the practitioner or an industrysupervisor for their ultimate judgment and decision.

In one embodiment, an imaging system with or without a bot having DFR,AI, and/or Metaverse, EON Reality, or Deloitte unlimited reality withits Dimension 10 studio, etc. software combined with or without afluidic lens camera (U.S. Pat. No. 9,016,860) can provide within a fewseconds multiple information, almost simultaneously, such as patient'srecognition, the visual aberration of the eye, correct the aberration,provide a prescription while the unit's optical coherence tomography(OCT) with an eye tracker and focusing system or another camera candiagnose, e.g., a normal cornea versus keratoconus, evaluate the statusof a cataract, determine presence or absence of diabetic retinopathy,age related macular degeneration in the retina (dry or wet), run apressure test (U.S. Pat. Nos. 10,736,571 and 7,828,440, which areincorporated by reference herein in their entireties) to evaluate manydifferent retinal parameters simultaneously, information or status ofthe optic nerve head, its variation from normal or existing perviousimages, thickness of peripapillary capillaries, or the central retina orthe peripheral retina in glaucoma, diabetic retinopathy, or presence orabsence of fluid or a tumor, presence or absence of a retinaldetachment, or macular hole, retinoschisis, inflammatory or a geneticdisease such as retinitis pigmentosa, glaucoma, etc., status of theretinal ganglion cells, the size and number of inner, and outer retinanuclear layer, status of the photoreceptors of the retina, sub-retinalfluid, intra-retinal fluid and differentiate it from a retinal orsub-retinal hemorrhage, status of the age related macular degeneration,or diagnose a disease process, etc., all in less than one minute, savingpatient and doctor's time and even provide suggestion of the follow upand treatment for a doctor to decide and confirm the findings of thedata or provide new medication(s), while maintaining the patientsecurity and confidentiality.

In one embodiment, an imaging system with or without a bot havingMetaverse, and DFR and AR or VR and 5G are especially useful forsurgeons in recognizing the patient, the operation site, which eye orextremity etc., thereby preventing operating on a wrong patient or wrongextremity or organ, while VR provides detailed, quality virtualdepictions of patients' anatomy presented in 3D anatomical images,improving doctors' surgical techniques or a technician in the industry,or a pilot, or a driver etc.

In one embodiment, an imaging system having dynamic facial recognition(DFR) with or without a bot is combined with artificial intelligencetechnology (AI) or EON reality for patients' recognition in 3-D imageformat from a near or far distance via internet communication, or at thetime of examination, with or without Metaverse which acts as virtualspace with all needed elements, doctor, patient, nurses, and/orindustrial or surgical tools with images, etc. that are not static andcan move in desired directions.

In one embodiment, an imaging system with or without a bot having DFRand AI not only images the facial structures and their subsequentdynamic changes, but the system is equipped with an optical coherencetomography (OCT) with near infrared wavelength that passes through theeye's structures such as cornea, lens, vitreous, retinal structuresincluding the retinal pigment epithelium, Bruch's membrane andsuperficial choroidal circulation and the larger veins underneath themand presents a visible, or computer-reconstructed cross-sectional twodimensional or 3-D image or live video of the circulation, from eachlayer of the cornea, lens, or retina, and the choroid circulation, etc.including the optic nerve head, thickness of superficial or deep retinalcapillaries around the optic nerve and in the central or peripheralretina, or the size of large arteries and veins (e.g., as described inPeyman remote laser delivery camera, U.S. Pat. No. 9,510,974, which isincorporated by reference herein in its entirety) with histologicalpresentation of each layer of the tissue as known in the art, so thatthe person or patient is recognized by his or her face or its dynamicchanges, but also by the microscopic structures of the cornea, lens,retina and the choroid and their thicknesses and their dynamic and notnecessarily static changes.

In one embodiment, an imaging system with or without a bot having theDFR camera records by its software and AI technology instant changescaused by a disease process or the potential changes that may happenwith time, such as hypertension, diabetes, aging, glaucoma, age relatedmacular degeneration, etc. in various layers or cellular or vascular ofthe retina, optic nerve, central or peripheral retina and/or instantrecognition of a disease process in a person, using the various camerasystems or preferentially the wide angle camera described in U.S. Pat.No. 9,510,974 or the fluidic lens camera (e.g., described in U.S. Pat.No. 9,016,860) or a fluidic light field camera, or a hyperspectralcamera where the camera is equipped with the desired light wavelengthemitted from one or more diode lasers or made visible byauto-fluorescence of the tissue or disease process, loss of the color indry form of age related macular degeneration, tissue oxygenation,localized or diffuse loss of the peripheral blood vessels in diabeticretinopathy, etc. and the changes of the retinal circulation or loss ofthe ganglion cells in glaucoma patients.

In one embodiment, an imaging system with or without a bot having DFRsoftware AI, Metaverse software (e.g., AR and VR in a virtual place,such as an office or operating room) combined with OCT or OCTangiography (OCTA) and a fluidic lens camera or wide angle camera (asdescribed in U.S. Pat. No. 9,510,974) and OCT and DFR software measuresand stores various parameters from the eye, cornea, lens, or retina OCTimages, encompassing the thickness of the retina, the nerve fibers,number of ganglion cells in the central, peripheral retina and the areaaround the optic nerve, the so-called circumpapillary area for adistance of 3 mm (2 degrees) beyond the optic nerve. And the retinalganglion cells or inner, and or outer nuclear layers of the retina andthe thickness of the inner and outer plexiform layer, inner segment andouter segment of the photoreceptors and their ellipsoid layer betweenthe inner and outer segment, retinal pigment epithelium, intra orsubretinal fluid thickness, Bruch's membrane and various layers ofchoroids using appropriate computer software that recognizes thesestructures in 2-3D under the normal intraocular pressure (IOP) and afterincreased IOP (measured by a pressure test) by certain degree of mmHg,e.g., 6 mmHg increased pressure to observe the changes, blanching,reduced thickness in the peripapillary capillaries, central retina orretinal periphery all documented with the OCT, or the camera can beequipped with a sensor chip that provides a fast, low cost diagnosis,automatically and recognizes the hyperspectral imaging, etc. if used andthe time to recovery of changes to its original appearance, using theDFR and AI software with a machine-learning algorithm such as anartificial neural network (ANN) or trained convolutional neural networkto image and analyze the changes recognized by dynamic recognitionimaging software, to diagnose the effect of the increased IOP pressureon the retina and the time taken to normalization (return of color,oxygenation, and thickness) after the pressure is released indicatingthe eye is normal or predisposed to glaucoma or its pressure iscontrolled by the medication.

In one embodiment, an imaging system with or without a bot having DFRsoftware, AI, Metaverse software (AR and VR in a virtual place, such asan office or operating room) combined with OCT or OCT angiography(OCTA), a fluidic lens camera and OCT and DFR software measures andstores various parameters from the eye, cornea, lens, or retina OCTimages, where the DFR serves as gate keeper for medical professionproviding individual security, or confidentiality associated with anyhealth care related personal issues and their managements including thehistory, physical and mental examination and/or the imaging and geneticdata to be secure and cannot be divulged without personal involvementand documented permission of the patient by dynamic facial recognitionwith its software and two stage procedure needing physical and mentalcollaboration of the patient to achieve the results with impliedpersonal or simultaneous documented agreement of the patient.

In one embodiment, an imaging system with or without a bot having DFRsoftware AI, Metaverse or EON reality software (AR and VR in a virtualplace, such as an office or operating room, factory, school, or shoppingmall) combines, measures, and stores various parameters and is similarto a virtual 3-D “drive through” by a “Google®” search street where DFRrepresents the key to open the door(s) to the person's private house,closets where participants can enter and look around carrying specialglasses, such as Google® glass, etc. or similarly, in operating room,the doctor can open a door virtually with DFR and AR to the surgicalroom and perform a procedure in 3-D by virtually entering a body organor an eye or brain, or intestinal tract, and look around for diagnosisor a disease process and either perform specific surgery in that area orobtain a biopsy from an area, e.g., a tumor or evaluate the state of thehealth of the tissue with hyperspectral imaging, OCT, or use a beam oflight, a diode or laser to treat an area as needed, or to diagnose andcompare the information with data through AI with previously obtainedinformation from other patients or even administer medication if neededor e.g., recognize and stop a bleeding vessel etc.

In one embodiment, an imaging system with or without a bot having theDFR and AI or machine-learning algorithm or an artificial neural network(ANN) with its imaging systems can diagnose localized or diffuse retinalcapillary loss, presence or absence of retinal micro-aneurysms, presenceor absence of localized or diffuse hemorrhage, presence or absence ofretinal traction with sub-retinal fluid or localized sub-foveal fluid,the degree of the loss of retinal capillaries or abnormalneovascularization in ARMD, etc., using OCTA in addition tohyperspectral imaging to diagnose presence of diabetic retinopathy,intra and sub-retinal fluid thickness, the degree of the damage to theretina, if the images are taken with fluidic fundus camera the systemautomatically also gives information on the visual acuity of the patientwho sees a visual display and the unit automatically corrects therefractive errors with the fluidic lenses simultaneously withoutpatients input in 1 to 2 seconds, when the system is equipped with OCT,it will provide an OCT and OCTA 2-3D images within seconds which isanalyzed with computer software, AI and an artificial neural network(ANN) or trained convolutional neural network (CNN).

In one embodiment, an imaging system with or without a bot at a firstexamination and any subsequent examinations, the patient is recognizedby DFR software and AI can evaluate structures such as retina, choroid,optic nerve, central and peripheral retina, the crystalline lens, corneaor glaucoma over a time period or by an initial exam images using amachine-learning algorithm, such as an artificial neural network (ANN)or trained convolutional neural network (CNN) to recognize patternswithin data by adjusting the strength of connections, or weights,between neurons or where the electronic circuit serves as both imagesensor and neural network for fast diagnosis that recognizes a pathologyin less than one second.

In one embodiment, the imaging camera with or without a bot is equippedwith an infrared diode in addition to a visible spectrum of lightvisible to the eye color (RGB), or invisible near infra-red formultispectral, or hyperspectral imaging camera providing rapidassessment of the observed pathology or compares it with the existingdata using DFR and AI, and machine-learning algorithm such as anartificial neural network (ANN) or trained convolutional neural network.

In one embodiment, an imaging system with or without a bot having DFRand AI and Metaverse is used for a population that does not have easyaccess to health personnel, with a shortage of facilities, doctors, orspecialists, for evaluation and data interpretation or lack of equipmentand/or medication, or lack of documentation of a patient's disease forcomparison and follow up, the system with DFR and AI and as anartificial neural network (ANN) or trained convolutional neural network(CNN), the examination, diagnosis, or treatment instructions can becarried out with ease and fast or one can use a modified smartphone witha DFR and AI algorithm for diagnosis and assisting the doctor withdecision making and treatment in a private confidential manner.Alternatively, the system can be used in banking transactions in avirtual office etc.

In one embodiment, an imaging system with or without a bot having facialrecognition (FR) is combined with an artificial neuronal network (ANN)equipped with a sensor chip to recognize automatically the patterns of aperson's face with its wrinkles and folds before and after a dynamicchange is induced on the face by frowning or smiling and subtracting theimages to observe dynamic changes in specific areas of the face usingDFR software with or without using 2D neural network image sensors(e.g., in sensor computing) by converting the photosensor network into aneural network, where the sensor acts as ANN creating machine vision orthe information is obtained from the retina using a fluid lens camera orfluidic light field camera (U.S. Pat. No. 10,606,066, which isincorporated by reference herein in its entirety) to obtain fast infocus 3-D images of a fluidic light field camera where the images areconverted to a digital format (analog-to-digital conversion) using amachine learning algorithm with (ANN) and trained convolutional neuralnetwork with a series of interconnected computational circles allowingthe network to classify images rapidly.

In one embodiment, a dynamic facial recognition (DFR) system with orwithout a bot combined with Metaverse along with optical coherenceangiography (OCT) and AI are used to evaluate the normal or variation ofan image, etc. over time (e.g., dynamic changes) of retinal thickness,population of the ganglion cells, density or superficial or deepcapillary plexus of the retina around the optic nerve, central andperipheral retina, presence or absence of cysts or fluid, status of thephotoreceptors layer, retinal pigment epithelium and choroidal capillaryplexus, in normal and inflammatory retinal diseases, such as age relatedmacular degeneration, diabetic retinopathy, wet form of age relatedmacular degeneration, or a genetic disease affecting central orperipheral retina to diagnose or treat them in an early stage.

In one embodiment, dynamic facial recognition (DFR) with or without abot combined with Metaverse along with optical coherence angiography(OCT) and AI to evaluate the variation from normal or variation overtime of (e.g., a dynamic change) of retinal thickness, population of theganglion cells, density of superficial or deep capillary plexus of theretina around the optic nerve, central and peripheral retina in variousstages of glaucoma or changes occurring as a result of increasing theintraocular pressure by applying pressure (e.g., during a pressure testdescribed in U.S. Pat. Nos. 10,736,571 and 7,828,440) to the eye toconfirm the diagnosis at an early stage disease, such as open angleglaucoma, or a corneal disease called keratoconus, a genetic disease, ordry eye before and after refractive surgery affecting the mechanicalstability of the cornea.

In one embodiment, an imaging system with or without a bot having adynamic recognition algorithm and trained convolutional neural networkwith artificial intelligence (AI) and/or augmented reality (AR), and/orvirtual reality (VR), and/or mixed reality technology (MR), and/orMetaverse, which is an extension of the virtual reality and/or augmentedreality, are used for imaging of the eye structures in 3-D (e.g., theretina), glaucomatous changes of the retina and optic nerve head orcloudiness of the crystalline lens to diagnose a cataract, posteriorlens capsule or macular edema after cataract surgery, thickness of thecornea, curvature of the cornea, its dioptric power, refractive error ofan eye, high and low order aberrations, documented as Zernike polynomialwith central corneal thickness for recognition of a disease process,such as keratoconus, etc. in an early stage of the disease and recommendtreatment, such as corneal crosslinking, etc. to the doctor.

In one embodiment, a DFR imaging system with or without a bot havingoptical coherence tomography (OCT), optical coherence angiography (OCTA)or color fundus photograph of the back of the eye (retina and choroidand optic nerve head) with artificial intelligence (AI) and/or augmentedreality (AR), and/or virtual reality (VR), and/or mixed realitytechnology (MR), and/or Metaverse and DFR algorithms and images used todiagnose immediate or long term dynamic changes of the retinalsuperficial capillary layers, deeper capillary layers of the retina,extent of the capillary perfusion of the fovea, size of the avascularfovea, thickness of the sensory retina, extent of thickness ofsuperficial retinal capillaries, hemorrhage at the optic nerve, positionand extent of the superficial and deep capillaries, the 3-D volumetriccharacteristics of the optic nerve head, thickness and dimension ofganglion cells, and inner, outer nuclear layer and photoreceptordimensions, etc. of the central and peripheral retina, presence or lossof retinal capillaries, presence or absence of hemorrhage, multimodal orhyperspectral imaging characteristics, of retinal circulation oroxygenation, or loss of vision when one increases the pressure of theeye mechanically (e.g., during a pressure test described in U.S. Pat.Nos. 10,736,571 and 7,828,440) with examination of the visual field,with simultaneous OCTA (angiography) of the optic nerve head and theretina creating a dynamic change in retinal, choroidal, and optic nervecapillaries and the time to recovery if instant pressure is applied fora few minutes and released to re-measure the described parameter to beevaluated with DFR and AI for the degree of changes and the extend of itto diagnose glaucoma and differentiate it from other diseases of theretina and predict the disease prognosis.

In one embodiment, an imaging system having the subtraction software ofDFR with or without a bot and AI is used to distinguish the changesoccurring rapidly by an increase in intraocular pressure or chronicvariation of the IOP in glaucoma or in a pressure test, where a definedpressure is applied to the eye and certain parameters are measured withOCT and OCTA to compare with initially obtained data, including thephysiological or functional changes, such as a diameter of the retinalartery or vein at the peripapillary capillaries, degree of theoxygenation change by mechanically applied force, and thickness ofperipapillary capillaries and time to recovery of the circulation andretinal oxygenation after applying a predetermined pressure to raise theintraocular pressure by 5-7 mm mercury to bleach the color of the fundusas a result of collapsing the retinal or choroidal vessels and theirnormal or slow recovery with dynamic mapping of the visual field duringthis test and their effect in normal versus a glaucoma patient withdelayed recovery leading to a more precise diagnosis and prediction inglaucoma.

In one embodiment, an imaging system having the OCTA equipment or otherimaging units are equipped with a motion tracking system and algorithmfor autofocusing as described by Peyman (e.g., as described U.S. Pat.Nos. 10,606,066, 9,191,568, 9,016,860, and/or 9,510,974, which areincorporated by reference herein in their entireties) is crucial forproper alignment of the OCT/OCTA, etc. and to evaluate results withsoftware having a facial recognition algorithm and AI, and complementedwith clinical information obtained with color photographs or multimodalimaging, e.g., for presence of the hemorrhage at the optic disc, oraneurysm, loss of capillaries, in the central and peripheral part of theretina, etc. in early stages or diabetic retinopathy, and in the centralversus branch vein occlusion, etc.

In one embodiment, a DFR imaging system with or without a bot for thevariation of the visual field is evaluated with software for facialrecognition algorithms, FR or DFR, and AI and Metaverse and AR, VR andMR with an automated visual field unit in which the patient confirmsseeing a projected light beam on various point of his or her retina(visual field) to confirm if he or she sees the light by pressing on abutton which indicates the degree of the visual field and records itslocation, increasing the intraocular pressure by certain amount e.g.,7-10 mm Hg pressure will affect if the patient sees all the spots oflight or how many of them in an area become invisible indicating thatthe increased IOP pressure even for a few minutes affected the functionof the retinal photoreceptors, and repeating the test after the pressurereturns to its original level indicates how long it has taken for thephotoreceptors to recover from the pressure test which is prolonged inglaucoma patients compared to the normal eye while the combination ofDFR software and AI or ANN or VR or AR or Metaverse can instantly recordand analyses the pattern of areas in which the visual field has beenaffected by, e.g., glaucoma or other diseases such as dry ARMD or indiabetic retinopathy or another disease process in a patient before fullmanifestation of the disease or have not yet become visible by thestandard photography, in essence measuring the physiological function ofthe retina at any desired area e.g., prepapillary (optic nerve), centralor peripheral retina to predict a disease process, such as glaucoma etc.

In one embodiment, an imaging system having a DFR and AI is used toincrease or decrease the intraocular pressure (e.g., as described in thepressure of U.S. Pat. Nos. 10,736,571 and 7,828,440) for predication ofglaucoma or retinal oxygenation.

In one embodiment, a DFR imaging system with or without a bot havingvariation of the central visual field evaluated with software of facialrecognition algorithms, DFR and AI and Metaverse and AR, VR, and MR withan automated visual field unit in which the patient confirms seeing aprojected light beam at various intensity on various points of thecentral retina to confirm that he or she is seeing the light by pressingon a button which indicates the degree of the visual field, increasingthe intraocular pressure by pressure test to certain amount e.g. 7-10 mmHG will affect if the patient sees all the spots of light or how many inwhich area become invisible indicating that the increased pressure evenfor a few minutes affected the function of the retinal photoreceptors,in the macular area, and repeating the test after the pressure returnsto its original level indicates how long it has taken for thephotoreceptors to recover after the pressure test which is prolonged inpatients with macular diseases, such as in diabetes, or dry form of AMDcompared to the normal eye, the combination of DFR software and AI orANN or VR or AR or deep learning-based (DL-based) images or Metaversecan instantly record and analyze the dynamic change that the testcreates in which the central vision has been affected by many diseasessuch as dry ARMD or in diabetic retinopathy, genetic diseases of theretina, or retinal findings and neurodegenerative and cardiovasculardiseases, such as Alzheimer's disease and hypertension in an early stagebefore full manifestation of the disease or has not yet become visibleby the standard photography, in essence measuring the physiologicalfunction of the macula.

In one embodiment, in an imaging system with or without a bot, thepressure test is replaced with a light test where the central or macularfunction is recorded by the lowest power of light beam that the patientcan see followed with shining a bright light at a certain power belowtoxic level of light, but high enough to blind the patient temporarilyfor a short time until the retinal photoreceptors recover to see theflash light blinking over the macular area and measure the time that ithas taken from the time that the eye was blinded to the time that thepatient sees the pulsating light again, analyzing the data with DFR andAI, Metaverse, and/or AI and AR and VR which correlates with thephysiological function of the central retinal area in normal maculararea which if prolonged, is predictive of the diseases affecting thephotoreceptors and vision as a whole in wet and dry form of ARMD (wet ordry), or another disease affecting the macula, etc. and the test can berepeated after therapy to evaluate the effect of a medication on thesedisease processes.

In one embodiment, the tests can be done with or without a chat botremotely via telemedicine with a DFR imaging, bot, or presence of adoctor or his technicians and Metaverse and/or AI using a laptop orsmartphone, etc.

In one embodiment, an imaging system with or without a bot having facialrecognition FR or DFR, and/or facial recognition software and deeplearning-based (DL-based) images and AI can predict if an eye is more orless prone to glaucoma by increasing the intra-ocular pressure, theso-called pressure test, beyond the normal 20 mm Hg., while recordingthe changes of the optic nerve head, such as increase of the optic nervecup depth, or change of blood oxygenation and the recovery time afterreleasing the external pressure, and changes in hyperspectral imagingand time to recovery, etc. are evaluated with combination of dynamicfacial recognition algorithm, deep neuronal network with AI andMetaverse with DFR subtraction software, simplifying the computeranalysis to provide a faster and more precise response if the patienthas glaucoma and the degree of the damage that is present.

Example 1

In one embodiment (DFR) software and deep learning-based (DL-based)images are used to show loss of superficial capillary vessel densitywithout loss of visual field over a period of time differentiatingglaucoma from the early stages of diabetic retinopathy.

Example 2

In another embodiment, the loss of the central visual field is observedusing a combination localized hyperspectral camera and DFR combined withAI to differentiate a localized arteriolar occlusion versus glaucoma,etc.

Example 3

A patient with normal intraocular pressure is studied with OCTA tomeasure the circumpapillary retinal nerve fiber layer (RNFL) thicknessand circumpapillary capillary density where DFR, assisted with a bot andAI finds loss of circu-papillary RNFL thickness and circumpapillarycapillary density indicating early stage glaucoma despite normal visualfield and normal intraocular pressure. The patient is suffering from lowtension glaucoma which was treated with topical allocation ofanti-glaucoma medication to lower the IOP below that level that existed.

In one embodiment, an imaging system using DFR with or without bot andMetaverse and AI and/or Metaverse, AR, VR and MR with a bot createsanimated speaking avatars of a physical selves permitting virtually toperform various interactions, such as assisting in surgery for properdifferentiation of fine structures, advising on the location,dimensions, image the tissues to identify and quantify areas ofinflammation and disease and to prevent mistakes in operating on wrongeye, or extremities or wrong patient which lead to the most commonlawsuits for malpractice after knee or eye surgery, etc. DFR technology,deep learning-based (DL-based) image and AI and bot can be used torecognize or communicate with the patient choosing the correct organ forsurgery or predict expected goal during surgery, while images areobtained with continuous feedback to the surgeon during the surgery insensitive areas, such as the eye, refractive surgery, cataract surgery,retinal surgery, heart, coronary artery stent or bypass, or peripheralvascular stents positioning, or brain surgery or vascular abnormalitiessuch as aneurisms with precise localization of an instrument, biopsy, orremoval of a tumor or in breast surgery with simultaneous imaging toremove a tumor completely for precise cosmetic replacement surgery or inprostate surgery avoiding damage to the nerves or gastrointestinalsurgery or orthopedic surgery or in emergency surgeries or roboticsurgery, etc. or detecting an disease or characterizing conditions usingreal-time clinical video imaging.

In one embodiment of DFR with a bot, AI and VR or Metaverse is used insurgery to remind the surgeon of steps of surgery and to avoid cuttingor penetrating sensitive area of the tissue potentially avoiding acomplication, etc.

In one embodiment of DFR with a bot, AI and VR and/or Metaverse is usedin remote surgery with laser or focused ultrasound, the bot can assistthe surgeon in the steps of surgery precise localization, and focusing alaser or the focused ultrasound to limit the damage to the specificlocation of a tumor, aneurysm, etc. and to avoid damaging the othernormal or sensitive area(s) of the tissue, e.g., in brain or spinalcord, eye tumor, prostate urinary tract, breast tumor or mouth tumor,tongue or through a tumor or the thyroid gland or ovary or kidneytumors, etc. potentially avoiding a complication, etc.

In one embodiment, warfare is terrible, but use of the weapons, guns,tanks, drones, bomb, etc. with an imaging system having DFR with orwithout a bot, and AI, AR and/or VR and/or Metaverse can avoidvictimizing the civilian population, recognizing the bystanders in dronewarfare who may be mistakenly taken for enemy, thereby reducing theinhumane casualties of war.

In one embodiment, an imaging system having been used in commerceencompasses DFR with or without a bot and AI, AR and/or VR and/orMetaverse and machine vision can avoid mistakes in delivery of goods, ormistakes in destination delivery, confirmation can be obtained when thepictures of the destination matches with the recorded destination priorto shipping, the technology is useful for human interface or machine orrobotic interface being more precise than presently available driverlesscars or drones for delivery of goods or establish a person dronecommunication by speaking or a person-car communication, etc., or in adriverless car, simply indicating (speaking) where the car is driving,speed, location, etc. outside temperature, or drone-person oralcommunication driving in a dark place or flying at night in a darkplace, etc. with GPS.

In one embodiment, for the ships at night, the DFR software and a botand drone see another ship or an object in the dark using IR for imagingand encrypted communication describing the object along with the imageor similarly drones can fly from a ship and guard the ship providinginformation miles away to the mothership or perform a task ifinstructed, the communication can be bilateral drone-person where theinformation is encrypted but when received converted to a voice andimage and recorded.

In one embodiment, the above concept can be used for mapping an area, orin a plane, or mounted on or in another system, such as missile that canchange course using another target as the communication with the centralcommand is established to change its direction under observation, thesystem can also indicate incoming missiles or it can sacrifice itself bystaying in the path of an incoming missile, etc. and other functionsknown in the art of defense, etc.

In one embodiment, a DFR imaging system with a bot having a combinationof various technology software that communicates with each other, suchas facial recognition and AI, AR, and/or VR and/or Metaverse, or ANN,CNN, or deep learning-based (DL-based) image or machine learning usingartificial neural networks (NN) permits writing a software capable oflearning from data in an autonomous fashion as either supervised orunsupervised learning where the algorithm is used to identify apattern-derived from the images making the doctor to patient'srelationship unhackable, since the software can be used simultaneouslyfor at least two parties for recognition through the internet, inaddition to confirming the location of the patient and the doctorthrough the GPS system, it can also potentially prevent cyberattackssince the data obtained with DFR is live and equipped with temperaturemeasurement using IR light and voice recognition software and live DFRthat produce an instantaneous confirmation or rejection from millions tobillions of information obtained from the images from the face or fromthe retina, providing the most reliable body temperature, along withlive voice recognition blocks the intruder and if combined with theinformation gained with the optical coherence tomography (OCT), thesystem not only record deeper surface layer, but also the retinalstructures beyond just superficial images providing billions ofinformation to confirm the identity of one or more people located on theother side of the system, e.g., a work place, a doctor's office,enterprise, with a patient, and consumer goods, pharmacy, etc. in avirtual session, the same software can also indicate the health of,e.g., an organ, an engine, a car, a bus, a plane or a ship, a submarine,or a building, or even software or a mathematic formula ordistinguishing a medication from a counterfeit medications or fakes,etc.

In one embodiment, an imaging system with or without a bot having thecombination of various technologies such as DFR and AI, AR or VR orMetaverse, or ANN, CNN used with various cameras including ahyperspectral camera can be used for recognition of a person or atransaction, e.g., with ATM machines or in banking and financialtransactions, or in the FDA, for approved medications, or by FBI, CIA,Home Security Department, or in the military or borders security, or inthe airports, etc.

In one embodiment, an imaging system with or without a bot has acombination of various technologies, such as DFR and AI, AR and/or VRand/or Metaverse or ANN, CNN used with various cameras including ahyperspectral camera can be used for recognition of animals of any kindor insects, snakes, or for food delivery, etc. by a drone etc.

In one embodiment, an imaging system, such as a fluidic lens camera(e.g., as described in U.S. Pat. Nos. 10,606,066, 9,191,568, and9,016,860) having DFR, with or without a bot, and AI, machine learningin an autonomous fashion as either supervised or unsupervised learningwith or without Metaverse or EON Reality combinations with anultra-resolution OCT having the potential to improve the detection ofsubclinical keratoconus or early keratoconus in routine ophthalmicpractice. The machine learning algorithm with the fluidic lens camera(e.g., as described in U.S. Pat. Nos. 10,606,066 and 9,191,568 and9,016,860) by doing the Zernike method, aberrations measurements orlower-order aberrations and HOAs (spherical aberration, coma, andtrefoil) and pachymetry looking for corneal apex displacement orposterior elevations, to create an NN classifier for discriminatingnormal, early keratoconus from later keratoconus or creating a heat mapof a subclinical keratoconus, depicting the curvature and elevation ofthe cornea.

Any of the features or attributes of the above described embodiments andvariations can be used in combination with any of the other features andattributes of the above described embodiments and variations as desired.

Without departing from the spirit and scope of this invention, one ofordinary skill can make various changes and modifications to theinvention to adapt it to various usages and conditions. As such, thesechanges and modifications are properly, equitably, and intended to be,within the full range of equivalence of the following claims.

What is claimed is:
 1. A telemedicine system with dynamic imaging,comprising: a local control system disposed at a first location and acentral control system disposed at a remote site, said remote site beingat a second location, said local control system being operativelycoupled to said central control system by means of a computer network;said local control system including a first computing device with afirst processor, a local control module, and a dynamic imaging system;said central control system including a second computing device with asecond processor and a remote control module; said dynamic imagingsystem including an imaging device operatively coupled to said firstcomputing device, said imaging device configured to capture images of abody portion of said patient over a predetermined duration of time sothat a displacement of said body portion of said patient is capable ofbeing tracked during said predetermined duration of time, said firstcomputing device being programmed to determine said displacement of saidbody portion of said patient over said predetermined duration of timeusing said captured images, and to compare said displacement of saidbody portion of said patient over said predetermined duration of time toa reference displacement of said body portion of said patient acquiredprior to said displacement so that dynamic changes in said body portionof said patient are capable of being assessed for the purpose ofidentifying said patient or evaluating physiological changes in saidbody portion.
 2. The telemedicine system according to claim 1, whereinsaid imaging device of said dynamic imaging system is in the form of alight field camera.
 3. The telemedicine system according to claim 1,wherein said dynamic imaging system is in the form of a dynamic facialrecognition system, and wherein said body portion of said patient forwhich said displacement is determined comprises a face or a portion ofthe face of said patient imaged in a two-dimensional orthree-dimensional manner.
 4. The telemedicine system according to claim3, wherein said first computing device is operatively coupled to achatbot that is configured to instruct the patient to perform a taskthat induces one or more detectable physical and physiological changesin said face or a portion of said face of said patient so that saidimaging device is able to capture images of said patient in differentphysical and/or physiological states.
 5. The telemedicine systemaccording to claim 1, wherein said imaging device of said dynamicimaging system is in the form of a multispectral camera or hyperspectralcamera configured to capture multispectral or hyperspectral images ofsaid body portion of said patient so that surface features andsubsurface features of said body portion of said patient are capable ofbeing analyzed.
 6. The telemedicine system according to claim 1, whereinsaid first computing device is operatively coupled to at least one of avirtual reality device, a mixed reality device, and an augmented realitydevice, said first computing device being programmed to enable saidpatient to communicate with a medical provider at said remote site usingsaid virtual reality device, said mixed reality device, and/or saidaugmented reality device so that medical provider is able to providevirtual medical care for said patient.
 7. The telemedicine systemaccording to claim 6, wherein said virtual reality device, said mixedreality device, and/or said augmented reality device enables the patientto be connected to a 3D network of virtual worlds containing healthinformation for said patient.
 8. The telemedicine system according toclaim 1, wherein said body portion of said patient being tracked by saidimaging device of said dynamic imaging system is affected by a diseaseprocess, and said first computing device is programmed to simulate aprogression of said disease process using artificial intelligence so asto supplement a standard imaging technique used to image said bodyportion of said patient affected by said disease process to diagnoseearly minute changes that occur in said body portion as a result of adisease process, said standard imaging technique consisting of X-ray,computed tomography (CT), magnetic resonance imaging (MRI), ultrasound,photoacoustic tomography, microscopy, optical coherence tomography(OCT), and confocal microscopy.
 9. The telemedicine system according toclaim 8, wherein said first computing device is operatively coupled to achatbot that is configured to receive voice input data from the patient,said first computing device being programmed to utilize the voice inputdata received from the patient to characterize said disease process ofthe patient using artificial intelligence.
 10. The telemedicine systemaccording to claim 8, wherein said disease process of the patientcomprises a disease process associated with an eye of the patient, saidimaging device of said dynamic imaging system is in the form of afluidic lens camera, and said first computing device is furtherprogrammed to diagnose said disease process associated with said eye ofthe patient by processing images obtained from said fluidic lens camerausing a machine learning algorithm.
 11. The telemedicine systemaccording to claim 1, wherein said first computing device is operativelycoupled to a visual display device, said first computing device beingprogrammed to generate and display a cognitive test on said visualdisplay device for assessing a mental ability of said patient, saidcognitive test comprising a plurality of different images for saidpatient to identify.
 12. The telemedicine system according to claim 1,wherein said plurality of different images of said cognitive testinclude images of at least one of the following: (i) one or morenumbers, (ii) one or more overlapping objects, (iii) well-knownpolitical figures or celebrities, and (iv) animals.